> As I understand AI "Training" (and I'm pretty much an idiot here) you have a > given neural net which is initially filled with hundreds of GB random data, > then you run input data through itand get the result. So I guess you input > a picture of a "bicycle" and it spits out "doggie", and you say "bad > computer", here is how you adjust your net values to becloser to the result > "bicycle".
Think of it like the very simplest pachinko machine: https://www.instructables.com/Pachinko-Machine/ …but typically very large. The “hundreds of GB of random data” are the _likelihoods_ of the ball going left or right at each junction. You drop a ball in somewhere along the top (the “somewhere” is the value of the input data), and it falls through the pins, and comes out somewhere at the bottom (that “somewhere” is the output data). When you’re _training_ the machine, you already know what a “good” output is, for any given input. So you drop the ball in the top, let it bounce its way threw off various pins, and see where it comes out on the bottom. If the output was “good,” you increase the weightings on each of the bounce directions of each of the pins it hit. Instead of 50%/50% left/right, if the ball went left, now it’s 51%/49%, for that pin. And so on for each pin that the ball hit on its way down. If the output was “bad,” you decrease the weightings for each pin/direction on the path traveled. Repeat many many times, and you wind up with a system which produces fairly replicable results. So, this is the “training” mechanism. It’s fairly complicated, and requires quite a bit of hardware and time. Now, take a snapshot of all of those probabilities and paths. Some paths have zero probability of being exercised, so you can trim those out of the model, making it smaller. Some paths have very low probability, so you can decide whether to prune them or not, depending how compact you need your model to be. Now you have a much smaller model. You can fit that in a very simple machine, which doesn’t need to understand the difference between “good” and “bad,” and doesn’t ever need to adjust any probabilities. Not only is it simple, it’s also fast and cheap. When you run it, it will always give you the same results with the same probabilities, but it will never “learn” anything new. This is “inference.” So, training requires very expensive hardware, and a lot of time. Like, thousands of these: https://www.ebay.com/sch/i.html?_from=R40&_nkw=nvidia+a100+80gb&_sacat=0&Brand=Nvdia&_dcat=27386&rt=nc&LH_ItemCondition=1000 https://www.nvidia.com/en-us/data-center/a100/ …for many weeks, for a large complicated model. So, training can cost tens or hundreds of millions of dollars, for big models. Inference can be done very inexpensively. If you have a recent Apple computer, you’ve already got excellent inference hardware built in. This has very good price/performance, if you don’t want to use an Apple: https://www.ebay.com/sch/i.html?_from=R40&_trksid=p2334524.m570.l1313&_nkw=nvidia+RTX+4090&_sacat=0&_odkw=geforce+RTX+4090&_osacat=0 If you want to buy something stand-alone, something like this works very well: https://www.amazon.com/NVIDIA-Jetson-Orin-64GB-Developer/dp/B0BYGB3WV4?th=1 Or at a smaller scale, this also has excellent price-performance, if you want to integrate it into a project, like a car: https://coral.ai/products/m2-accelerator-dual-edgetpu/ But the real price/performance killer is the Apple Mac Studio with the 76-core GPU and 192gb of RAM: https://www.apple.com/shop/buy-mac/mac-studio/24-core-cpu-60-core-gpu-32-core-neural-engine-64gb-memory-1tb# So, inference can be done with good price/performance anywhere from $20 to $6600, mostly depending on the size of the model (amount of RAM to hold it in memory) you want to load. -Bill _______________________________________________ Address messages to ev@lists.evdl.org No other addresses in TO and CC fields HELP: http://www.evdl.org/help/
