Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

I'm constantly surprised how many people are critical of research to understand neural nets, immediately telling me they are black boxes and hopeless to understand. I believe it's a consequence of being portrayed as the opposite of (classically interpretable) linear regression.

Many people additionally have little patience for research when the engineering is moving so quickly. Even many interpretability researchers give up far too soon if research doesn't yield immediately gratifying results.



I'm not in the field but I think it's because historically neural nets were looked down and deemed unpromising because they lacked understanding, compared to Symbolic AI or SVM for example. Since the Deep Learning revolution, which is engineering driven, the trend has inverted, research to understand and theory are seen as the things that hindered progress with neural nets in the past.


Part of the issue with neural nets is that historically they were next to impossible to train. ADAM, BatchNorm/LayerNorm, initialization schemes, and GPUs for pure speed really helped to change all of that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: