76 lines
2.8 KiB
Markdown
76 lines
2.8 KiB
Markdown
|
AI LLM inference evaluations tokens slop
|
||
|
|
|
||
|
|
## Terminology
|
||
|
|
|
||
|
|
- Do you know what an LLM is? a brief introduction
|
||
|
|
- neural networks
|
||
|
|
- a stateless machine
|
||
|
|
- the memory problem (what is a context window)
|
||
|
|
- outdated by definition
|
||
|
|
- lacks the "world" model, doesn't comprehend consequences
|
||
|
|
- harness to support training
|
||
|
|
- harness to support inference
|
||
|
|
|
||
|
|
## What is vibe-coding?
|
||
|
|
|
||
|
|
- if human is not in the loop it's vibe coding
|
||
|
|
- never look at the generated code
|
||
|
|
- just use strong, iterative prompting
|
||
|
|
|
||
|
|
What are the problems:
|
||
|
|
|
||
|
|
- allucinations (provide an example)
|
||
|
|
- maintainability (provide an example)
|
||
|
|
- security (provide an example)
|
||
|
|
- IP related risks
|
||
|
|
- individual cognitive risks
|
||
|
|
- team cognitive load
|
||
|
|
|
||
|
|
https://www.youtube.com/shorts/aIvHf8vsWBM ([Ilya Sutskever](https://en.wikipedia.org/wiki/Ilya_Sutskever) interview short extract about vibecoding)
|
||
|
|
https://www.youtube.com/watch?v=5vp9ypOUgMw [Theo - tr.gg, Vie coding is for Senior Developes](https://www.youtube.com/watch?v=5vp9ypOUgMw)
|
||
|
|
|
||
|
|
Engineers are so dramatic!
|
||
|
|
|
||
|
|
## Automatic coding
|
||
|
|
|
||
|
|
Show different levels of interaction with coding agents:
|
||
|
|
|
||
|
|
for each, mention limitations
|
||
|
|
|
||
|
|
- Phase 1: automatic code completion
|
||
|
|
- Phase 2: copy-paste from a chat prompt
|
||
|
|
- Phase 3: chat agent integrated in the IDE
|
||
|
|
- Phase 4: multi-agent orchestration
|
||
|
|
- Phase 5: full Spec-driven development (still a myth)
|
||
|
|
|
||
|
|
## Demo Claude code
|
||
|
|
|
||
|
|
- skills: Opticodds fixtures
|
||
|
|
- hooks: avoid exposing credentials to context
|
||
|
|
|
||
|
|
## The AI SLOP: are we ready to deploy to production without reviews?
|
||
|
|
|
||
|
|
without any specific discipline, AI slop will "help" in the following ways:
|
||
|
|
|
||
|
|
- inbalance between code creation and code review
|
||
|
|
- noise is being introduced and consistency is quickly lost
|
||
|
|
- team specific expertise will decline
|
||
|
|
|
||
|
|
## What a wise AI/coding strategy looks like then?
|
||
|
|
|
||
|
|
- let's take a look at open source project maintainers (what are they doing)?
|
||
|
|
- they are adding friction on the PR triaging and review
|
||
|
|
- no more breadcrumbs: always include a link to the conversation
|
||
|
|
|
||
|
|
## My completely unrequested piece of advice
|
||
|
|
|
||
|
|
- New tools require new manners: stop sloppypasta (https://stopsloppypasta.ai/en/)
|
||
|
|
- Reviews are the last gatekeeping tool to avoid the slop - not taking reviews seriously = doomed
|
||
|
|
- If you don't understand something, you should not use automatic programming to "solve" it
|
||
|
|
- prediction: I believe in the following months/years there will be a bloom of indivisuals and organizations that will tackle engineering problems with no engineering background
|
||
|
|
- Use all the new tools, experiment and profit
|
||
|
|
- Your ability to read, understand, and produce code is still valuable (more that many might think) - excercise these skills
|
||
|
|
- link to the tsoding channel (include some screenshots from youtube)
|
||
|
|
- link to the codingtraing channel (include some screenshots from youtube)
|
||
|
|
- build pet projects and leverage coding agents for exploration
|