2.8 KiB
2.8 KiB
AI LLM inference evaluations tokens slop
Terminology
- Do you know what an LLM is? a brief introduction
- neural networks
- a stateless machine
- the memory problem (what is a context window)
- outdated by definition
- lacks the "world" model, doesn't comprehend consequences
- harness to support training
- harness to support inference
What is vibe-coding?
- if human is not in the loop it's vibe coding
- never look at the generated code
- just use strong, iterative prompting
What are the problems:
- allucinations (provide an example)
- maintainability (provide an example)
- security (provide an example)
- IP related risks
- individual cognitive risks
- team cognitive load
https://www.youtube.com/shorts/aIvHf8vsWBM (Ilya Sutskever interview short extract about vibecoding) https://www.youtube.com/watch?v=5vp9ypOUgMw Theo - tr.gg, Vie coding is for Senior Developes
Engineers are so dramatic!
Automatic coding
Show different levels of interaction with coding agents:
for each, mention limitations
- Phase 1: automatic code completion
- Phase 2: copy-paste from a chat prompt
- Phase 3: chat agent integrated in the IDE
- Phase 4: multi-agent orchestration
- Phase 5: full Spec-driven development (still a myth)
Demo Claude code
- skills: Opticodds fixtures
- hooks: avoid exposing credentials to context
The AI SLOP: are we ready to deploy to production without reviews?
without any specific discipline, AI slop will "help" in the following ways:
- inbalance between code creation and code review
- noise is being introduced and consistency is quickly lost
- team specific expertise will decline
What a wise AI/coding strategy looks like then?
- let's take a look at open source project maintainers (what are they doing)?
- they are adding friction on the PR triaging and review
- no more breadcrumbs: always include a link to the conversation
My completely unrequested piece of advice
- New tools require new manners: stop sloppypasta (https://stopsloppypasta.ai/en/)
- Reviews are the last gatekeeping tool to avoid the slop - not taking reviews seriously = doomed
- If you don't understand something, you should not use automatic programming to "solve" it
- prediction: I believe in the following months/years there will be a bloom of indivisuals and organizations that will tackle engineering problems with no engineering background
- Use all the new tools, experiment and profit
- Your ability to read, understand, and produce code is still valuable (more that many might think) - excercise these skills
- link to the tsoding channel (include some screenshots from youtube)
- link to the codingtraing channel (include some screenshots from youtube)
- build pet projects and leverage coding agents for exploration