• 0 Posts
  • 611 Comments
Joined 2 years ago
cake
Cake day: June 21st, 2023

help-circle



  • The conclusion of this experiment is objectively wrong when generalized. At work, to my disappointment, we have been trying for years to make this work, and it has been failure after failure (and I wish we’d just stop, but eventually we moved to more useful stuff like building tools adjacent to the problem, which is honestly the only reason I stuck around).

    There are a couple reasons why this problem cannot succeed:

    1. The outputs of LLMs are nondeterministic. Most problems require determinism. For example, REST API standards require idempotency from some kinds of requests, and a LLM without a fixed seed and a temperature of 0 will return different responses at least some of the time.
    2. Most real-world problems are not simple input-output machines. When calling, let’s say for example, an API to post a message to Lemmy, that endpoint does a lot of work. It needs to store the message in the darabase, federate the message, and verify that the message is safe. It also needs to validate the user’s credential before all of this, and it needs to record telemetry for observability purposes. LLMs are not able to do all this. They might, if you’re really lucky, be able to generate code that does this, but a single LLM call can’t do it by itself.
    3. Some real world problems operate on unbounded input sizes. Context sizes are constrained and as currently designed cannot handle unbounded inputs. See signal processing for an example of this, and for an example of a problem a LLM cannot solve because it cannot receive the input.
    4. LLM outputs cannot be deterministically improved. You can make changes to prompts and so on but the output will not monotonically improve when doing this. Improving one result often means sacrificing another result.
    5. The kinds of models you want to run are not in your control. Using Claude? K Anthropic updated the model and now your outputs all changed and you need to update your prompts again. This fucked us over many times.

    The list keeps going on. My suggestion? Just don’t. You’ll spend less time implementing the thing than trying to get an LLM to do it. You’ll save operating expenses. You’ll be less of an asshole.






  • What they’re doing should be outright illegal in most countries; it’s equivalent to changing a contract unilaterally after both parties signed it.

    Update to [COMPANY NAME]'s Policies

    Yes, this should be illegal, but it’s already common practice. I’m just hoping that enough of this will eventually get people to stop buying these products, and hopefully we can start seeing some real legislation against it in some countries.

    Additionally, I’d strongly advise against buying any sort of “smart” device, unless you’re pretty sure the benefits of connecting your toaster to the internet outweighs all the risks.

    This should be obvious at this point. “Smart” just means “internet-connected”, and we already know what happens to every device that connects to a remote server during regular operation: telemetry (and not the nice debugging kind but the “what do you use” kind), and advertisements.

    Including corporations and crackers

    The “crackers” part of this confuses me. Samsung is a Korean company. The chairman’s name is Lee Jae-yong (이재용). Samsung NA’s CEO is Yoonie Joung. Maybe I’m misreading this?





  • To me, as an American, having a young president might even be more newsworthy.

    We don’t know what presidents do behind closed doors. What I do know is what some of that demographic that I know has done behind closed doors, and let’s just say I’d be shocked if any future president was truly the first one to be gay (or at least not completely straight).

    As far as NL goes, I love it there. Genuinely nobody I interacted with cared at all how you identify. They just want to get it right. I think the biggest issue I’ve seen there is regarding transgender people, but the scale isn’t even on the same planet as it is in the US, and reaching that point would be a monumental milestone here worthy of a national holiday.





  • Used Claude 4 for something at work (not much of a choice here and that team said they generate all their code). It’s sycophantic af. Between “you’re absolutely right” and it confidently making stuff up, I’ve wasted 20 minutes and an unknown number of tokens on it generating a non-functional unit test and then failing to solve the type errors and eslint errors.

    There are some times it was faster to use, sure, but only because I don’t have the time to learn the APIs myself due to having to deliver an entire feature in a week by myself (rest of the team doesn’t know frontend) and other shitty high level management decisions.

    At the end of the day, I learned nothing by using it, the tests pass but I have no clue if they test the right edge cases, and I guess I get to merge my code and never work on this project again.