Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
@lungdart proof-of-concept. It's not multi-layer as far as hidden layers go. It's a single layer and that's the goal.
I started out trying to confirm a few different questions:
1. Can we train and perform complex functions under circumstances traditionally considered impossible: the answer it turns out is yes (using multiple biases per node)
2. Can we train a network without backprop. In the single hidden layer example it turns out the answer is yes. Jury is out of it works across multiple hidden layers.
3. Can we train a model to perform even more complex functions in a hidden layer? That's what I'm working on now. -
@atheist the hidden layer in the model has one node instead of two. Adding multiple biases instead of one per node (essentially acting as free parameters) and doing a nonstandard alternative to backprop makes the network separable even with a single hidden layer with only one node.
-
@atheist no, haven't pub'd the code yet.
Focused on general improvements, training the model to perform more complex functions the hidden layer, and handling inputs that aren't just binary.
After that I want to train it to do something non-trivial, clean up the code, write a proper article maybe on substack or something, and seek a proper venue to present it and the backprop alternative.
Significantly tired of toiling in obscurity broke as a joke. Not after riches, just enough to continue further research. Hard to hold down a 9-5, find time to do deepwork and keep a family. Pick two as they say.
Spent a lot of time trying to find others willing to assist or help but mostly concluded I'd have to either go work at a research lab (fat chance w/o a b/g and a degree), or get to a place where I could pay others to work with me.
Lot of low-hanging fruit at this time, more than I can handle by myself frankly.
Machine Learning messed up!
When your CPU is motivated and gives more than his 100%
What is machine learning?
research 10.09.2024
I successfully wrote a model verifier for xor. So now I know it is in fact working, and the thing is doing what was previously deemed impossible, calculating xor on a single hidden layer.
Also made it generalized, so I can verify it for any type of binary function.
The next step would be to see if I can either train for combinations of logical operators (or+xor, and+not, or+not, xor+and+..., etc) or chain the verifiers.
If I can it means I can train models that perform combinations of logical operations with only one hidden layer.
Also wrote a version that can sum a binary vector every time but I still have
to write a verification table for that.
If chaining verifiers or training a model to perform compound functions of multiple operations is possible, I want to see about writing models that can do neighborhood max pooling themselves in the hidden layer, or other nontrivial operations.
Lastly I need to adapt the algorithm to work with values other than binary, so that means divorcing the clamp function from the entire system. In fact I want to turn the clamp and activation into a type of bias, so a network
that can learn to do binary operations can also automatically learn to do non-binary functions as well.
random
machine learning
ml