That would probably be implementing multithreading in shell scripts.

https://gitlab.com/netikras/bthread

The idea (though not the project itself) was born back when I still was a sysadmin. Maintaining 30k servers 24/7 was quite something for a team of merely ~14 people. That includes 1st line support as well.

So I built a script to automate most of my BAU chores. You could feed a list of servers - tens or hundreds or more - and execute the same action on each of them (actions could be custom or predefined in the list of templates). Neither Puppet nor Chef or Ansible or anything of sorts was consistently deployed in that zoo, not to mention the corp processes made use of those tools even a slower approach than the manual one, so I needed my own solution.

The problem was the timing. I needed all those commands to execute on all the servers. However, as you might expect, some servers could be frozen, others could be in DMZ, some could be long decommed (and not removed from the listings), etc. And these buggars would cause my solution to freeze for longer than I'd like. Not to mention that running something like `sar -q 1 10` on 200 servers is quite time-consuming itself :)

And how do I get that output neatly and consistently (not something you'd easily get with moving the task to a background with '&'. And even with that you would not know when are all the iterations complete!)?

So many challenges...

I started building the threading solution that would

- execute all the tasks in parallel
- do not write anything to disks
- assign a title to each of the tasks
- wait for all the tasks to complete in either
> the same sequence as started
> as soon as the task finishes

- keep track of each task's
> return code
> output
> command
> sequence ID
> title

- execute post-finish actions (e.g. print to the console) for each of the tasks -- all the tracked properties are to be accessible by the post-finish actions.

The biggest challenges were:
a) how do I collect all that output without trashing my filesystems?
b) how do I synchronize all those tasks
c) how do I make the inception possible (threads creating threads that create their own threads and so on).

Took me some time, but I finally got there and created the libbthread library. It utilizes file descriptors, subshells and some piping magic to concentrate the output while keeping track of all the tasks' properties. I now use it extensively in my new tools - the ones where I can't use already existing tools and can't use higher-level languages.

Just created a tiny (not yet stable) selector library that allows you to change properties on multiple elements just like you would do on a single one. It also has some more useful features. https://github.com/kosimst/kazel
Still in development, but prototype should work. What do you think?

Follow-up on https://devrant.com/rants/5001553/...

How the fuck are Jupyter notebooks so popular in research? Like some dude had an idea to take perfectly good markdown and python code, add a whole lot of transitional properties to make version control impossible, encode it as JSON on the assumption that a human could somehow look at it and make sense of countless escaped characters and base64 encoded data, create dedicated software people need to install in order to read what used to be simple plain text, and think "This. This is what 99% of data researchers will use from now on." And somehow, overwhelming majority of researchers agreed that this extremely inefficient data format is the best there is and they should develop all their tools around it.

Pretty much Python automation on steroids.

https://github.com/konradhalas/...

Dacite is an integral part in it, cause it makes most auto generated API wrappers like Cloudflare API "maintenable".

Getting dicts, converting via Dacite to defined data classes...

Then using TOML to define e.g. output parameters (e.g. list of classes / properties one needs)...

... To export them via Pandas to anything what one needs.

It's just so comfortable.

Definining data classes, sprinkle the API calls and dacite in it, some definition via TOML, done.

Yes, lots of dark vodoo / behind the scenes magic... But ... It removes all this annoying fucked up boilerplate writing that takes ages and makes it frustrating.

As long as the data wrapper (e.g. clousflare API) generates Dicts, its really minutes to get an export working.

If you know the pain of having to deal with multiple accounts, different formats (e.g. different companies)… hours of manual copy pasting to aggregate the data etc.

Then you can maybe understand why I love this so much.

Data classes and dacite makes painful confusing workflows so much nicer and self documenting, I get wet in my pants while writing this. :)

Keeping up the tradition!
https://devrant.com/rants/15030806/...

Now powered by the awesome Claude 4!!!! The latest bleeding edge gem in the LLM trend that by the bold claims of big tech and various youtubers is stunning and will replace programmers/insane/godsmacking

This time, I decided to post because I was on the verge of a mental breakdown and I had a firsthand experience and a bit of free time, so instead of waiting that the rage boils down, I just took the chance and so we have a fresh AI experience to proof my previous rants.

Problem:
I have an application that manages interactions between a Mediator pattern between Kafka, some http listeners and other stuff (Rabbitmq, Redis (redis= another shit) etc etc)

Kafka handling was already done, i needed to exacly replicate what was being done with kafka for http. I said, heck, we have AI, it's good to do bovine stupid work, why not let him copy the whole structure, they have also agents on copilot on vs code, and they pulled out Claude 4, let's give it faith again.

Big error.
Beside the havoc the fucking agentic AI not being able to fully copy an already existing pattern and scaffolding, it did half shit wrong.

One of the most frustrating parts, since today I have an example, was this (summarized version of the prompt because I was very much more specific also giving the AI files and pointers to the code and explaining it):

PROMPT:"I have to return fake http transports, just copy what exists for kafka and adapt it to a new transport type called "Http", this is the structure {json with the class}

AI: "I need to see the current ITransportsItemResponse interface first to understand the structure, then I'll modify ...... BLAH BLAH"

Of course I had to review the whole code.

PROMPT:"any valid reason why you dumb shithead didn't respect the IGetKafkaConnectionsItemResponse and generic IGetTransportsItemResponse format and you pulled out of your ass a full blown IGetHttpConnectionsItemResponse with extra properties when compared to IGetKafkaConnectionsItemResponse"

AI: "You're absolutely right, I fucked up. Looking at the IGetHttpConnectionsItemResponse interface, it has way more properties than the basic IGetTransportsItemResponse and IGetKafkaConnectionsItemResponse interfaces. BLAH BLAH wall of text reasoning"

Then it modified the api calls to suit the kafka objects and didn't update any class of the Http transports

PROMPT:"So you tought it was a good idea to leave the #sym:IGetHttpConnectionsItemResponse like it is respect to #sym:IGetKafkaConnectionsItemResponse ? Why? And answer the why don't apologize like a little wuss"

AI: "I left them different because they represent different domain concepts... BLAH BLAH EXPLANATIONS WALL OF TEXT"

Now ai tought it was good to do this and that but....

PROMPT: "In your fucking careful analysis didn't you reallize the "items" object are a basic representation and in the fuckin same folder there is also the object with the full set of properties?"

Literally it missed objects used in the same folder for different scopes and modified stuff without caring.

AI: "You're absolutely right. I missed that there are TWO different response types BLA BLAH"

I won't continue to not get too lenghty than it already is but the point is:

AI IS RETARDED.
People say it will replace programmers.
People says agents are the future.

Sad reality it's an overglorified broken ball of if/else that can't do shit well beside bovine work.

No amount of tutoring it with careful prompts, explainig the code and whatever else is going to fix it.
I've used gpt since gpt 3 and no model has been up to anything good, not even NLP. They suck also at the sole scope they were invented for.

I tried to ask GPT to make a curriculum based on another, I gave it the example curriculum and another one with the informations.

I carefully explained that it must not be a copy of the other, they are 2 different roles and to play by fantasy to make it look it was written by 2 different persons and to not copy stuff from the other.

Hope lost. It looked like the other curriculum was copied over and some words swapped, lol.

What a fucking joke, lmao, I am studying deep learning and machine learning to get on the bandwagon to make my professional figure more appealing, but I can already feel this is a waste of time.

{
"$schema": "http://json-schema.org/draft-04/...#",
"type": "object",
"id": "https://[URL_NAME]/forms/{id}/...#",
"properties": {
"title" : { "type": "string" },
"date" : { "type": "string" },
"content" : { "type": "string" },
"date_start": { "type": "string" },
"date_end" : { "type": "string" },
"status" : {
"type" : "string",
"enum" : ["1", "2", "3", "4", "5"]
}
},
required [
"title",
"date",
"content",
"date_start",
"date_end",
"status"
]
}

See if you can notice the error is this schema. Don't copy and paste it. I change some format to obsfucate the real data naming, but this schema error is still up there

Just wasted my 30 minutes staring at this

After the fun I had with the XEN Orchestrator UI ( https://devrant.com/rants/2554182/ ) I build an exporter that normalizes XEN / Proxmox API output and writes it into a nice spreadsheet.

Took PHP 7.4 for a spin. Sweet jesus, lot of nice stuff.

Been nearly a year since I did something larger than small scripts in PHP, but felt really at home again.

The type hinting and arrow functions made writing the exporter a breeze.

DTOs with typed properties spared me quite a bit of headache when normalizing the different APIs...

Utilizing *sort with fn arrow function is a pretty nice and concise one liner with the spaceship operator.

And I have now a nice spreadsheet...

Thanks at the PHP folks.