I’ve used spicy auto-complete, as well as agents running in my IDE, in my CLI, or on GitHub’s server-side. I’ve been experimenting enough with LLM/AI-driven programming to have an opinion on it. And it kind of sucks.
(1) boilerplate code that is so predictable a machine can do it
The thing I hate most about it is that we should be putting effort into removing the need for boilerplate. Generating it with a non-deterministic 3rd party black box is insane.
Hard disagree. There is a certain level of boilerplate that is necessary for an app to do everything it needs. Django, for example, requires you to specify model files, admin files, view files, form files, etc. that all look quite similar but are dependent on your specific use case. You can easily have an AI write these boilerplate for you because they are strongly related to one another, but they can’t easily be distilled down to something simpler because there are key decisions that need specified.
All of that can be automated with tools built for the task. None of this is actually that hard to solve at all. We should automate away pain points instead of boiling the world in the hopes that a linguistic, stochastic model can just so happen to accurately predictively generate the tokens you want in order to save a few fucking hours.
The hubris around this whole topic is astounding to me.
I think you underestimate the amount of business logic contained in boilerplate. (Or maybe we’re just talking about different definitions of what boilerplate is). LLMs can understand that business need while most code generators cannot.
LLMs do not understand anything. There is no semantic understanding whatsoever. It is merely stochastic generation of tokens according to a probability distribution derived from linguistic correlations in its training data.
Also, it is incredibly common for engineers at businesses to have their engineers write code to automate away boilerplate and otherwise inefficient processes. Nowhere did I say that automation must always be done via open source tooling (though that is certainly preferable when possible, of course).
What do you think people and businesses were doing before all of this LLM insanity? Exactly what I’m describing. It’s hardly novel or even interesting.
OK sure if you want to be pedantic. The point is that LLMs can do things traditional code generators can’t.
You don’t have to like it or use it. I myself am very vocal about the weaknesses and existential dangers of AI code. It’s going to cause the worst security nightmares in humanity’s recorded history. I recommend to companies that they DON’T trust LLMs for their coding because it creates unmaintainable nightmares of spaghetti code.
But pretending that they have NO advantages over traditional code generators is utter silliness perpetuated by people who refuse to argue in good faith.
Because it’s not worth inventing a whole tool for a one-time use. Maybe you’re the kind of person who has to spin up 20 similar Django projects a year and it would be valuable to you.
But for the average person, it’s far more efficient to just have an LLM kick out the first 90% of the boilerplate and code up the last 10% themself.
I’d rather use some tool bundled with the framework that outputs code that is up to the current standards and patterns than a tool that will pull defunct patterns from it’s training data, make shit up, and make mistakes that easily missed by a reviewer glazing over it
I honestly don’t think such a generic tool is possible, at least in a Django context. The boilerplate is about as minimal as is possible while still maintaining the flexibility to build anything.
If it’s as minimal as possible, then the responsible play is to write it thoughtfully and intentionally rather than have something that can make subtle errors to slip through reviews.
Almost all my projects have the same kind of setup nowadays. But thats just work. For personal projects, I use a subset-ish. Theres a custom Admin module that I use to make ALL classes into Django admin models and it takes one import, boom done.
“Not worth inventing”? Do you have any idea how insanely expensive LLMs are to run? All for a problem whose solution is basically static text with a few replacements?
You’re focused too much on the “inventing” and not enough on the “one time”. A flexible solution can find value even if it’s otherwise inferior to a rigid one.
If it’s 90% boilerplate like you were saying above, how flexible does it need to be, really? If it only needs to get 90% there, surely a general-purpose scaffolding tool could do the job just as well.
You could use a snippet engine or templates with your editor, but unless you get a lot of reuse out of them, it’s probably easier and quicker to use an LLM for the boilerplate.
Easier and quicker, but finding subtle errors in what looks like it should be extremely hard to fuck up code because someone used an LLM for it is getting really fucking old already, and I shudder at all the things like that are surely being missed. “It will be reviewed” is obviously not sufficient
The thing I hate most about it is that we should be putting effort into removing the need for boilerplate. Generating it with a non-deterministic 3rd party black box is insane.
Hard disagree. There is a certain level of boilerplate that is necessary for an app to do everything it needs. Django, for example, requires you to specify model files, admin files, view files, form files, etc. that all look quite similar but are dependent on your specific use case. You can easily have an AI write these boilerplate for you because they are strongly related to one another, but they can’t easily be distilled down to something simpler because there are key decisions that need specified.
All of that can be automated with tools built for the task. None of this is actually that hard to solve at all. We should automate away pain points instead of boiling the world in the hopes that a linguistic, stochastic model can just so happen to accurately predictively generate the tokens you want in order to save a few fucking hours.
The hubris around this whole topic is astounding to me.
I think you underestimate the amount of business logic contained in boilerplate. (Or maybe we’re just talking about different definitions of what boilerplate is). LLMs can understand that business need while most code generators cannot.
LLMs do not understand anything. There is no semantic understanding whatsoever. It is merely stochastic generation of tokens according to a probability distribution derived from linguistic correlations in its training data.
Also, it is incredibly common for engineers at businesses to have their engineers write code to automate away boilerplate and otherwise inefficient processes. Nowhere did I say that automation must always be done via open source tooling (though that is certainly preferable when possible, of course).
What do you think people and businesses were doing before all of this LLM insanity? Exactly what I’m describing. It’s hardly novel or even interesting.
OK sure if you want to be pedantic. The point is that LLMs can do things traditional code generators can’t.
You don’t have to like it or use it. I myself am very vocal about the weaknesses and existential dangers of AI code. It’s going to cause the worst security nightmares in humanity’s recorded history. I recommend to companies that they DON’T trust LLMs for their coding because it creates unmaintainable nightmares of spaghetti code.
But pretending that they have NO advantages over traditional code generators is utter silliness perpetuated by people who refuse to argue in good faith.
Why does it have to be AI instead of a purpose built, deterministic tool?
Because it’s not worth inventing a whole tool for a one-time use. Maybe you’re the kind of person who has to spin up 20 similar Django projects a year and it would be valuable to you.
But for the average person, it’s far more efficient to just have an LLM kick out the first 90% of the boilerplate and code up the last 10% themself.
I’d rather use some tool bundled with the framework that outputs code that is up to the current standards and patterns than a tool that will pull defunct patterns from it’s training data, make shit up, and make mistakes that easily missed by a reviewer glazing over it
I honestly don’t think such a generic tool is possible, at least in a Django context. The boilerplate is about as minimal as is possible while still maintaining the flexibility to build anything.
If it’s as minimal as possible, then the responsible play is to write it thoughtfully and intentionally rather than have something that can make subtle errors to slip through reviews.
I just use https://github.com/cookiecutter/cookiecutter and call it a day. No AI required. Probably saves me a good 4 hours in the beginning of each project.
Almost all my projects have the same kind of setup nowadays. But thats just work. For personal projects, I use a subset-ish. Theres a custom Admin module that I use to make ALL classes into Django admin models and it takes one import, boom done.
Sure, I’ve used that too in the past.
“Not worth inventing”? Do you have any idea how insanely expensive LLMs are to run? All for a problem whose solution is basically static text with a few replacements?
You’re focused too much on the “inventing” and not enough on the “one time”. A flexible solution can find value even if it’s otherwise inferior to a rigid one.
If it’s 90% boilerplate like you were saying above, how flexible does it need to be, really? If it only needs to get 90% there, surely a general-purpose scaffolding tool could do the job just as well.
Back in the day, I used CakePHP to build websites, and it had a tool that could “bake” all the boilerplate code.
You could use a snippet engine or templates with your editor, but unless you get a lot of reuse out of them, it’s probably easier and quicker to use an LLM for the boilerplate.
Easier and quicker, but finding subtle errors in what looks like it should be extremely hard to fuck up code because someone used an LLM for it is getting really fucking old already, and I shudder at all the things like that are surely being missed. “It will be reviewed” is obviously not sufficient
Is it possible to use deterministic automation for some boilerplate instead of LLMs?
Sure but it’s a lot less flexible. As much hate as they get, LLMs are the best natural language processors we have. By FAR.
Code is not natural language.
No, but the business requirements obviously are. Code does not exist in a vacuum.