Nostr Web Client

Working on a new AI/ML project partly to learn but also to see if something interesting will arise from "scratch."

A missing piece this morning is that of a reward system. I don't know how to tell this thing what's good and what's bad programatically. Seems like the perfect time to "do things that don't scale" and execute the reward function myself. I figure I'll let it ask me after each step for 👍 or 👎. An app would be cool so it could just be taps but I'm a cli-guy so maybe I'll go with vim bindings.

Reply to this note

Please Login to reply.

Discussion

paulo 2y ago

What kind of predictions are you training your model on?

STERRY 2y ago

To start it'll be word association. I prototyped a network with a set of mindmap-like files but want to start with nothing but a tokenizer and train it to associate appropriate words.

paulo 2y ago

So it’s going to be predicting words within a particular context window.

You could measure how similar the embeddings for a particular prediction are to the embeddings of the ground truth word. Like an autoencoder.

You would use something like MSE for the loss function.

STERRY 2y ago

Allow me to rephrase to see if I understand. The embeddings refer to an internal state (like a vector representing activation levels of neurons) reached after feeding in the words in the context window.

So you're saying I could compare the state which creates a prediction to the state that is achieved by inputting only the predicted word?

paulo 2y ago

Yes, you have an internal representation of the inputs in a continuous semantic space, but you can also have a final layer that outputs an approximation for the embeddings of the predicted word. And you can measure how similar that approximation is to the ground truth.

Alternatively you can also index all the possible outputs and represent them as integers, in which case you would be making descrete predictions. For this you could use sparse categorical cross entropy in your loss function.