[D] How to differentiate between your idea or implementation being wrong?

Simple: just train a classifier on historical examples of projects that did and didn't work out.

Unit test unique functions with simple cases.

You can also generate trivially simple data or only train on a few samples. Any modern deep net should overfit and learn to perfectly categorize them.

You mention, but #1 suggestion is to work with other people. Collaboration, consideration, review, and critique are the way. You will learn and debug at the same time.

As meat-based-algorithms we are filled with biases and the propensity for errors. Work together with differently-biased people. We also make errors in at least both categories you suggest. Bring different people together to debug through perspective diversity. The combined de-correlated basket of biases and perspectives will help you in more ways than just telling you if it is your idea or your code.

There may also be no answer to why it doesn't work on this 2-case "it didn't work" labelling task you set the reddit network. {idea, impl, both, neither, something else 1, ...}. For example, you didn't let it run long enough, or the dataset was insufficient (which you may call implementation I guess).

When you are creating your implementation, ensure that you can over-fit to tiny data before burning real (C|G|T)PU time. If you can't do that, the implementation is wrong (at least).

Just keep running experiments and changing things until you notice your inevitable mistakes.

My supervisor gave me some really good advice. I was working on a vision related project and I was getting really good loss performance on a relatively simple network. I panicked cause I thought I might have messed up and he told me to just visualize some predictions and see if they match my understanding of the task I was training for. I immediately saw what was going wrong. Other than that, I also realized that I could have saved a lot of time if I had set up rudimentary unit tests for checking the behavior of certain components/functions in my code similar to what has been suggested in other comments.

Combine point 1 and point 2. Break down your work into manageable units that can be tested via toy examples, then get your collaborators\peers to look at your tests and test results.

You have to make it easy for others if you want their help. Asking them to deep dive into your code is not the way.

Unit tests for individual modules with simple input <> output pairs. For example, if I'm implementing the quadratic formula, I'd have a few hand calculated input output test cases.

On an unrelated note, never use the quadratic formula on a computer due to numerical stability issues.

Know some math?

Try to think of a way to implement your idea in a different but mathematically equivalent way and then compare the results.

For example, you could use a naive, explicit implementation instead of a more numerically stable log softmax or convert code processing inputs in parallel to loop through them one at a time instead.

Sometimes you can uniformly randomize some input. If then outputs show unexpected patterns or symmetries, the implementation can be wrong.

Look for unexplainable behavior. If you can't explain something that happens that's likely or bug or gross mathematical error.