I agree with you here (and also cool project/idea). What I’m trying to get at with my analogy is that, if you understand a programme’s source code, you understand and are able to predict its behaviour in a meaningful way (although of course you lose some amount of knowledge of what the processor exactly does with which registers etc). When you only know the prompt used to generate a programme, you can’t do that. That’s why you have to manually check the code an LLM writes, but not the assembly generated by a compiler.
I agree with you here (and also cool project/idea). What I’m trying to get at with my analogy is that, if you understand a programme’s source code, you understand and are able to predict its behaviour in a meaningful way (although of course you lose some amount of knowledge of what the processor exactly does with which registers etc). When you only know the prompt used to generate a programme, you can’t do that. That’s why you have to manually check the code an LLM writes, but not the assembly generated by a compiler.