Re: [FRIAM] Deep learning training material

[Steve Smith]

The "celebration of the hand" project (coordinated with UNM's Maxwell 
museum of anthropology) is nearly 15 years defunct now. We made a tiny 
bit of progress on some LANL small-business-assistance time from an ML 
researcher, a small bit of seed-funding and matching time on my own 
part.   The key to this was the newly available prosumer-grade 
laser-scanners of the time, as well as emerging photogrammetric techniques.


The point of the project was to augment what humans were already 
doing...  both pointing to similarities too subtle for a human to notice 
or based on dimensions a human expert wouldn't be able to identify 
directly.   I haven't touched bases on where that field has gone in most 
of that intervening time.   My paleontological/archaelogical partner in 
that endeavor went his own way (he was more interested in developing a 
lifestyle business for himself than actually applying advanced 
techniques to the problem at "hand" (pun acknowledged)) and then a few 
years later he died which is part of the reason for not (re)visiting it 
myself...


The relevant issue to the linked article(s) is mostly that we were 
building modelless models (model-free learning) from the data with the 
hope (assumption?) that *some* of the features that might emerge as 
being good correlates to related/identical "hands" might be ones that 
humans could detect or make sense of themselves. The state of the art at 
the time was definitely an "art" and was based on the expertise of 
contemporary flint knappers trying to reproduce the patterns found in 
non-contemporary artifacts.   We did not do any actual work (just 
speculation guided background research) on pottery and textiles...  the 
Maxwell museum researcher we worked with was also "overcome by events" 
simply needing to develop the exhibits and not having time/focus to 
attend to the more researchy aspects...  her interest was more on 
textiles than ceramics or lithics... which seemed to be the area we 
likely had the least advantage over human-experts.


My niece works at the U of Az Archaelogy dept cataloging the 
accellerating number of artifacts coming in.   At some point I can 
imagine an automated classification system taking over much of the 
"mundane" aspects of this work...   like a self-driving car that at 
least provides collision avoidance and lane following...



On 1/9/23 10:29 AM, Pieter Steenekamp wrote:
Thanks for the references, I've briefly looked at them and am looking 
forward to perusing them more closely. The interpretation of ML is a 
big thing of course. The machine gives you the results and it would 
sometimes be nice if it can be accompanied by some explanation of how 
it achieved it.


What you describe about your work sounds very interesting indeed. My 
gut feel is that, at least with the current generation of AI, human 
ingenuity and judgement would be far superior to AI in recognising 
similarities in the "hand" of the artisans involved. But AI could well 
be a powerful tool in assisting the humans?


On Mon, 9 Jan 2023 at 18:50, Steve Smith  wrote:

I was hoping this article would have more meat to it, but the main
point seems highly relevant to a practical/introductory workshop
such as the one you are developing:

The Need for Interpretable Features: Taxonomy and Motivation



https://scitechdaily.com/mit-taxonomy-helps-build-explainability-into-the-components-of-machine-learning-models/

My limited experience with ML is that the convo/invo-lutions that
developers go through to make their learning models work well tend
to obscure the interpretability of the results.   Some of this
seems unavoidable (inevitable?), particularly the highly technical
reserved terminology that often exists in a given domain, but this
team purports to help provide guidelines for minimizing the
consequences...

My main experience with this domain involved an attempt to find
distance measures between the flake patterns in 3D scanned
archaeological artifacts, starting with lithics but also aspiring
to work with pottery and textiles... essentially trying to
recognize similarities in the "hand" of the artisans involved.


On 1/9/23 1:29 AM, Pieter Steenekamp wrote:

@Russ,

a) I appreciate the suggestion to include a simple neural network
that can make predictions based on inputs and be trained using
steepest descent to optimize its weights. This would be a
valuable addition to my training material, as it provides a
foundational understanding of how neural networks work.

b) My focus is on providing practical training for professionals
with deep domain knowledge but limited software experience, who
are looking to utilize deep learning as a tool in their
respective fields. In contrast, it seems that your focus is on
understanding the inner workings of deep learning. Both
approaches have 

Re: [FRIAM] Sorting Algorithm? AI? Identifying "types" within data

[Nicholas Thompson]

To my uneducated eye, this seemed like one of Jon’s problems.Sent from my Dumb PhoneOn Jan 7, 2023, at 6:23 AM, Frank Wimberly  wrote:This answer seems reasonable to me.  I worked on Project Talent during 1967 which had some similar goals and data.  Seehttps://en.m.wikipedia.org/wiki/Project_TalentOur data was for thousands of highschool students and our software was all written in Fortran.---Frank C. Wimberly140 Calle Ojo Feliz, Santa Fe, NM 87505505 670-9918Santa Fe, NMOn Fri, Jan 6, 2023, 11:32 PM Pieter Steenekamp  wrote:I asked https://chat.openai.com/chat and here is the conversation:Pieter Steenekampcan you suggest a solution for the following problem "I'm hoping someone here could help out. Let's imagine I had some data where each row was a person's career. We could list major events every year.For example: 2004 they were highered, 2007 they get a promotion, 2010 they leave for a different company, 2012 they come back at a higher level, 2015 get a promotion, then no change until 2022.  Let's say I had data like this for roughly 2 million people, and that there are around 10 different types of changes that could happen during any time period (could be yearly, quarterly, monthly, I can make it how I want).  I was hoping we could ask a computer to tell us if there were "types of careers" that people had. We could say "put all these careers into 4 buckets" or "7 buckets" based on similarity. Then we could look at the piles the computer made and try to make sense of them. One type might be "company man" for people who tend to stay in place for 20 or more years, another type could be a "rotator", who leaves and returns every 3 years or so. Etc. The point is, I want a computer to make the piles for me, rather than trying to come up with potential piles a priori. Are there methods for doing this? I know it's a problem we've *talked* about a lot, but I don't know if there are solutions. " https://chat.openai.com/chatThere are several approaches that you could use to group the careers of these 2 million people into clusters based on similarity. One approach is to use unsupervised machine learning techniques, specifically clustering algorithms, to automatically group the data into clusters. Some popular clustering algorithms that you could use for this problem include k-means, hierarchical clustering, and DBSCAN (Density-Based Spatial Clustering of Applications with Noise). You could also use dimensionality reduction techniques, such as PCA (Principal Component Analysis) or t-SNE (t-Distributed Stochastic Neighbor Embedding), to reduce the dimensionality of the data and then use a clustering algorithm to group the data into clusters. Another approach is to use supervised machine learning techniques, where you would need to manually label a subset of the data with the cluster labels, and then use this labeled data to train a classifier to predict the cluster labels for the remaining data. It's worth noting that these approaches will require some feature engineering, where you will need to extract relevant features from the data that can be used to represent the careers of the people in the dataset. Once you have identified and extracted these features, you can then use them as input to the machine learning algorithms to group the data into clusters.Pieter Steenekampwould it then be possible to get  for example if there were "types of careers" that people had?https://chat.openai.com/chatYes, once you have grouped the careers of the 2 million people into clusters using one of the approaches I mentioned above, you can then analyze the characteristics of each cluster to identify common patterns and trends within each cluster. This will allow you to identify "types of careers" that people have, based on the similarity of their career events. For example, if one of the clusters is characterized by people who have had long tenures at a single company, you could label this cluster as "company man" careers. Similarly, if another cluster is characterized by people who frequently switch between companies, you could label this cluster as "rotator" careers. You can then use this information to gain insights into the types of careers that are represented in the dataset, and potentially draw conclusions about the factors that may influence career paths and success.Pieter Steenekampcan you propose python code to do sometingh like this pleasehttps://chat.openai.com/chatSure, here is an example of how you could use the scikit-learn library in Python to group the careers of the 2 million people into clusters using the k-means algorithm:Copy codefrom sklearn.cluster import KMeans# Load the data into a NumPy arraydata = ""># Create the k-means modelmodel = KMeans(n_clusters=4)# Fit the model to the datamodel.fit(data)# Predict the cluster labels for each data pointlabels = model.predict(data)# Print the cluster labelsprint(labels)This code will group the data into 4 clusters, as specified by the n_clus

Re: [FRIAM] Deep learning training material

[Russ Abbott]

Like Python, Lisp is type-checked at runtime. Python does have an
optional static
type system . It's been a
while since I worked with Haskell, but at the time I really appreciated the
static typing. It seemed that once I got something to compile, it was
almost guaranteed to work properly. I haven't used Python's static type
"suggestions" recently. As I remember, it was useful and informative, but
on balance, it made the code so much more cluttered that I stopped using it.

-- Russ

On Sun, Jan 8, 2023 at 11:20 AM Marcus Daniels  wrote:

> The main defects of both R and Python are a lack of a typing system and
> high performance compilation.  I find R still follows (is used by) the
> statistics research community more than Python.   Common Lisp was always
> better than either.
>
> Sent from my iPhone
>
> On Jan 8, 2023, at 11:03 AM, Russ Abbott  wrote:
>
> 
> As indicated in my original reply, my interest in this project grows from
> my relative ignorance of Deep Learning. My career has focussed exclusively
> on symbolic computing. I've worked with and taught (a) functional
> programming, logic programming, and related issues in advanced Python; (b)
> complex systems, agent-based modeling, genetic algorithms, and related
> evolutionary processes, (c) a bit of constraint programming, especially in
> MiniZinc, and (d) reinforcement learning as Q-learning, which is
> reinforcement learning without neural nets. I've always avoided neural
> nets--and more generally numerical programming of any sort.
>
> Deep learning has produced so many impressive results that I've decided to
> devote much of my retirement life to learning about it. I retired at the
> end of Spring 2022 and (after a break) am now devoting much of my time to
> learning more about Deep Neural Nets. So far, I've dipped my brain into it
> at various points. I think I've learned a fair amount. For example,
>
>- I now know how to build a neural net (NN) that adds two numbers
>using a single layer with a single neuron. It's really quite simple and is,
>I think, a beautiful example of how NNs work. If I were to teach an intro
>to NNs I'd start with this.
>- I've gone through the Kaggle Deep Learning sequence mentioned
>earlier.
>- I found a paper that shows how you can approximate
>any differentiable function to any degree of accuracy with a single-layer
>NN. (This is a very nice result, although I believe it's not used
>explicitly in building serious Deep NN systems.)
>- From what I've seen so far, most serious DNNs are built using Keras
>rather than PyTorch.
>- I've looked at Jeremy Howard's fast.ai material. I was going to go
>through the course but stopped when I found that it uses PyTorch. Also, it
>seems to be built on fast.ai libraries that do a lot of the work for
>you without explanation.  And it seems to focus almost exclusively on
>Convolutional NNs.
>- My impression of DNNs is that to a great extent they are *ad hoc*.
>There is no good way to determine the best architecture to use for a given
>problem. By architecture, I mean the number of layers, the number of
>neurons in each layer, the types of layers, the activation functions to
>use, etc.
>- All DNNs that I've seen use Python as code glue rather than R or
>some other language. I like Python--so I'm pleased with that.
>- To build serious NNs one should learn the Python libraries Numpy
>(array manipulation) and Pandas (data processing). Numpy especially seems
>to be used for virtually all DNNs that I've seen.
>- Keras and probably PyTorch include a number of special-purpose
>neurons and layers that can be included in one's DNN. These include: a
>DropOut layer, LSTM (short-long-term memory) neurons, convolutional layers,
>recurrent neural net layers (RNN), and more recently transformers, which
>get credit for ChatGPT and related programs. My impression is that these
>special-purpose layers are *ad hoc* in the same sense that functions
>or libraries that one finds useful in a programming language are *ad
>hoc*. They have been very important for the success of DNNs, but they
>came into existence because people invented them in the same way that
>people invented useful functions and libraries.
>- NN libraries also include a menagerie of activation functions. An
>activation function acts as the final control on the output of a layer.
>Different activation functions are used for different purposes. To be
>successful in building a DNN, one must understand what those activation
>functions do for you and which ones to use.
>- I'm especially interested in DNNs that use reinforcement learning.
>That's because the first DNN work that impressed me was DeepMind's DNNs
>that learned to play Atari games--and then Go, etc. An important advantage
>of Reinforcement Learning (RL)

Re: [FRIAM] Deep learning training material

[Pieter Steenekamp]

Thanks for the references, I've briefly looked at them and am looking
forward to perusing them more closely. The interpretation of ML is a big
thing of course. The machine gives you the results and it would sometimes
be nice if it can be accompanied by some explanation of how it achieved it.

What you describe about your work sounds very interesting indeed. My gut
feel is that, at least with the current generation of AI, human ingenuity
and judgement would be far superior to AI in recognising similarities in
the "hand" of the artisans involved. But AI could well be a powerful tool
in assisting the humans?

On Mon, 9 Jan 2023 at 18:50, Steve Smith  wrote:

> I was hoping this article would have more meat to it, but the main point
> seems highly relevant to a practical/introductory workshop such as the one
> you are developing:
>
> The Need for Interpretable Features: Taxonomy and Motivation
> 
>
>
> https://scitechdaily.com/mit-taxonomy-helps-build-explainability-into-the-components-of-machine-learning-models/
>
> My limited experience with ML is that the convo/invo-lutions that
> developers go through to make their learning models work well tend to
> obscure the interpretability of the results.   Some of this seems
> unavoidable (inevitable?), particularly the highly technical reserved
> terminology that often exists in a given domain, but this team purports to
> help provide guidelines for minimizing the consequences...
>
> My main experience with this domain involved an attempt to find distance
> measures between the flake patterns in 3D scanned archaeological artifacts,
> starting with lithics but also aspiring to work with pottery and
> textiles...  essentially trying to recognize similarities in the "hand" of
> the artisans involved.
>
>
> On 1/9/23 1:29 AM, Pieter Steenekamp wrote:
>
> @Russ,
>
> a) I appreciate the suggestion to include a simple neural network that can
> make predictions based on inputs and be trained using steepest descent to
> optimize its weights. This would be a valuable addition to my training
> material, as it provides a foundational understanding of how neural
> networks work.
>
> b) My focus is on providing practical training for professionals with deep
> domain knowledge but limited software experience, who are looking to
> utilize deep learning as a tool in their respective fields. In contrast, it
> seems that your focus is on understanding the inner workings of deep
> learning. Both approaches have their own merits, and it is important to
> cater to the needs and goals of different learners.
>
> Pieter
>
> On Sun, 8 Jan 2023 at 21:20, Marcus Daniels  wrote:
>
>> The main defects of both R and Python are a lack of a typing system and
>> high performance compilation.  I find R still follows (is used by) the
>> statistics research community more than Python.   Common Lisp was always
>> better than either.
>>
>> Sent from my iPhone
>>
>> On Jan 8, 2023, at 11:03 AM, Russ Abbott  wrote:
>>
>> 
>> As indicated in my original reply, my interest in this project grows from
>> my relative ignorance of Deep Learning. My career has focussed exclusively
>> on symbolic computing. I've worked with and taught (a) functional
>> programming, logic programming, and related issues in advanced Python; (b)
>> complex systems, agent-based modeling, genetic algorithms, and related
>> evolutionary processes, (c) a bit of constraint programming, especially in
>> MiniZinc, and (d) reinforcement learning as Q-learning, which is
>> reinforcement learning without neural nets. I've always avoided neural
>> nets--and more generally numerical programming of any sort.
>>
>> Deep learning has produced so many impressive results that I've decided
>> to devote much of my retirement life to learning about it. I retired at the
>> end of Spring 2022 and (after a break) am now devoting much of my time to
>> learning more about Deep Neural Nets. So far, I've dipped my brain into it
>> at various points. I think I've learned a fair amount. For example,
>>
>>- I now know how to build a neural net (NN) that adds two numbers
>>using a single layer with a single neuron. It's really quite simple and 
>> is,
>>I think, a beautiful example of how NNs work. If I were to teach an intro
>>to NNs I'd start with this.
>>- I've gone through the Kaggle Deep Learning sequence mentioned
>>earlier.
>>- I found a paper that shows how you can approximate
>>any differentiable function to any degree of accuracy with a single-layer
>>NN. (This is a very nice result, although I believe it's not used
>>explicitly in building serious Deep NN systems.)
>>- From what I've seen so far, most serious DNNs are built using Keras
>>rather than PyTorch.
>>- I've looked at Jeremy Howard's fast.ai material. I was going to go
>>through the course but stopped when I found that it uses PyTorch. Also, it
>>seems to be built on fast.ai libraries t

Re: [FRIAM] Deep learning training material

[Steve Smith]

I was hoping this article would have more meat to it, but the main point 
seems highly relevant to a practical/introductory workshop such as the 
one you are developing:


The Need for Interpretable Features: Taxonomy and Motivation


https://scitechdaily.com/mit-taxonomy-helps-build-explainability-into-the-components-of-machine-learning-models/

My limited experience with ML is that the convo/invo-lutions that 
developers go through to make their learning models work well tend to 
obscure the interpretability of the results.   Some of this seems 
unavoidable (inevitable?), particularly the highly technical reserved 
terminology that often exists in a given domain, but this team purports 
to help provide guidelines for minimizing the consequences...


My main experience with this domain involved an attempt to find distance 
measures between the flake patterns in 3D scanned archaeological 
artifacts, starting with lithics but also aspiring to work with pottery 
and textiles...  essentially trying to recognize similarities in the 
"hand" of the artisans involved.



On 1/9/23 1:29 AM, Pieter Steenekamp wrote:

@Russ,

a) I appreciate the suggestion to include a simple neural network that 
can make predictions based on inputs and be trained using steepest 
descent to optimize its weights. This would be a valuable addition to 
my training material, as it provides a foundational understanding of 
how neural networks work.


b) My focus is on providing practical training for professionals with 
deep domain knowledge but limited software experience, who are looking 
to utilize deep learning as a tool in their respective fields. In 
contrast, it seems that your focus is on understanding the inner 
workings of deep learning. Both approaches have their own merits, and 
it is important to cater to the needs and goals of different learners.


Pieter

On Sun, 8 Jan 2023 at 21:20, Marcus Daniels  wrote:

The main defects of both R and Python are a lack of a typing
system and high performance compilation.  I find R still follows
(is used by) the statistics research community more than Python.  
Common Lisp was always better than either.

Sent from my iPhone


On Jan 8, 2023, at 11:03 AM, Russ Abbott 
wrote:


As indicated in my original reply, my interest in this project
grows from my relative ignorance of Deep Learning. My career has
focussed exclusively on symbolic computing. I've worked with and
taught (a) functional programming, logic programming, and related
issues in advanced Python; (b) complex systems, agent-based
modeling, genetic algorithms, and related evolutionary processes,
(c) a bit of constraint programming, especially in MiniZinc, and
(d) reinforcement learning as Q-learning, which is reinforcement
learning without neural nets. I've always avoided neural
nets--and more generally numerical programming of any sort.

Deep learning has produced so many impressive results that I've
decided to devote much of my retirement life to learning about
it. I retired at the end of Spring 2022 and (after a break) am
now devoting much of my time to learning more about Deep Neural
Nets. So far, I've dipped my brain into it at various points. I
think I've learned a fair amount. For example,

  * I now know how to build a neural net (NN) that adds two
numbers using a single layer with a single neuron. It's
really quite simple and is, I think, a beautiful example of
how NNs work. If I were to teach an intro to NNs I'd start
with this.
  * I've gone through the Kaggle Deep Learning sequence mentioned
earlier.
  * I found a paper that shows how you can approximate
any differentiable function to any degree of accuracy with a
single-layer NN. (This is a very nice result, although I
believe it's not used explicitly in building serious Deep NN
systems.)
  * From what I've seen so far, most serious DNNs are built using
Keras rather than PyTorch.
  * I've looked at Jeremy Howard's fast.ai 
material. I was going to go through the course but stopped
when I found that it uses PyTorch. Also, it seems to be built
on fast.ai  libraries that do a lot of the
work for you without explanation.  And it seems to focus
almost exclusively on Convolutional NNs.
  * My impression of DNNs is that to a great extent they are /ad
hoc/. There is no good way to determine the best architecture
to use for a given problem. By architecture, I mean the
number of layers, the number of neurons in each layer, the
types of layers, the activation functions to use, etc.
  * All DNNs that I've seen use Python as code glue rather than R
or some other language. I like Python--so I'm pleased with that.
  * T

Re: [FRIAM] Sorting Algorithm? AI? Identifying "types" within data

[Eric Charles]

>From what I can tell "one-hot encoding" is just another term for dummy
coding the data, i.e., make it a bunch of 1/0 columns. H2o seems more
promising, but seems to require a backbone of quantitative data that you
can substitute (based on something akin to a regression) for the
categorical variables.

It seems weird to me that this doesn't exist. It seems like such a basic
thing to want the computer to do for you. The lack of this reinforces my
belief that a ton of AI work is just people who don't understand basic
analytic techniques trying hard not to learn how to do them. If most of the
AI efforts are just giving me the results of a regression, that's not super
exciting when I am comfortable running regressions myself.





On Sat, Jan 7, 2023 at 3:30 PM Pieter Steenekamp 
wrote:

> One way to handle categorical input data for machine learning is to
> convert it using one-hot encoding - it's not difficult but a bit cumbersome.
> Fortunately there are other options. H2O is a machine learning library
> available in both Python and R that does this conversion "under the hood".
>
> I asked ChatGPT about "machine learning with categorical inputs using
> H2O", she answered:
>
> H2O is a popular open-source library for training machine learning models
> in Python and R. It is designed to be scalable, fast, and easy to use, and
> it includes a number of algorithms for classification, regression, and
> clustering.
>
> H2O can handle categorical variables natively, so you don't need to use
> one-hot encoding or any other special techniques to work with them. When
> you import your data into H2O, you can specify which columns are
> categorical, and H2O will take care of the rest.
>
> Here is an example of how you might use H2O to train a machine learning
> model with categorical inputs in Python:
>
> Copy code
> import h2o
> # Initialize H2O
> h2o.init()
> # Import your data into H2O
> df = h2o.import_file("path/to/your/data.csv")
> # Specify which columns are categorical
> df['input_variable'] = df['input_variable'].asfactor()
> # Train a model using the categorical input_variable column
> model = h2o.H2ORandomForestEstimator(categorical_encoding =
> "one_hot_explicit")
> model.train(["input_variable"], "output_variable", df)
> Here is an example of how you might use H2O to train a machine learning
> model with categorical inputs in R:
> Copy code
> library(h2o)
> # Initialize H2O
> h2o.init()
> # Import your data into H2O
> df <- h2o.importFile("path/to/your/data.csv")
> # Specify which columns are categorical
> df[, "input_variable"] <- as.factor(df[, "input_variable"])
> # Train a model using the categorical input_variable column
> model <- h2o.randomForest(x = "input_variable", y = "output_variable",
> training_frame = df)
>
> On Sat, 7 Jan 2023 at 17:37, Eric Charles 
> wrote:
>
>> That's somewhat helpful. Having looked up several of these
>> algorithms (I'm still checking a few), it seems like they all input some
>> sort of distance measure between the items (analogous to the distance
>> between their coordinates on a cartesian graph), and then do some sort of
>> distance-minimization function. The challenge here is that I don't have
>> anything equivalent to that: The data is primarily categorical.
>>
>> Does anyone on here actually have experience doing that kind of work?
>>
>> It's not that it would be impossible for me to change the categorical
>> data into something more quantitative, but doing so would bake in my
>> assumptions about how the categories should be determined.
>> 
>>
>>
>> On Sat, Jan 7, 2023 at 1:32 AM Pieter Steenekamp <
>> piet...@randcontrols.co.za> wrote:
>>
>>> I asked https://chat.openai.com/chat and here is the conversation:
>>>
>>> *Pieter Steenekamp*
>>> can you suggest a solution for the following problem "I'm hoping someone
>>> here could help out. Let's imagine I had some data where each row was a
>>> person's career. We could list major events every year.For example: 2004
>>> they were highered, 2007 they get a promotion, 2010 they leave for a
>>> different company, 2012 they come back at a higher level, 2015 get a
>>> promotion, then no change until 2022.  Let's say I had data like this for
>>> roughly 2 million people, and that there are around 10 different types of
>>> changes that could happen during any time period (could be yearly,
>>> quarterly, monthly, I can make it how I want).  I was hoping we could ask a
>>> computer to tell us if there were "types of careers" that people had. We
>>> could say "put all these careers into 4 buckets" or "7 buckets" based on
>>> similarity. Then we could look at the piles the computer made and try to
>>> make sense of them. One type might be "company man" for people who tend to
>>> stay in place for 20 or more years, another type could be a "rotator", who
>>> leaves and returns every 3 years or so. Etc. The point is, I want a
>>> computer to make the piles for me, rather than trying to come up with
>>> potential piles a priori. Are

Re: [FRIAM] Deep learning training material

[Pieter Steenekamp]

@Russ,

a) I appreciate the suggestion to include a simple neural network that can
make predictions based on inputs and be trained using steepest descent to
optimize its weights. This would be a valuable addition to my training
material, as it provides a foundational understanding of how neural
networks work.

b) My focus is on providing practical training for professionals with deep
domain knowledge but limited software experience, who are looking to
utilize deep learning as a tool in their respective fields. In contrast, it
seems that your focus is on understanding the inner workings of deep
learning. Both approaches have their own merits, and it is important to
cater to the needs and goals of different learners.

Pieter

On Sun, 8 Jan 2023 at 21:20, Marcus Daniels  wrote:

> The main defects of both R and Python are a lack of a typing system and
> high performance compilation.  I find R still follows (is used by) the
> statistics research community more than Python.   Common Lisp was always
> better than either.
>
> Sent from my iPhone
>
> On Jan 8, 2023, at 11:03 AM, Russ Abbott  wrote:
>
> 
> As indicated in my original reply, my interest in this project grows from
> my relative ignorance of Deep Learning. My career has focussed exclusively
> on symbolic computing. I've worked with and taught (a) functional
> programming, logic programming, and related issues in advanced Python; (b)
> complex systems, agent-based modeling, genetic algorithms, and related
> evolutionary processes, (c) a bit of constraint programming, especially in
> MiniZinc, and (d) reinforcement learning as Q-learning, which is
> reinforcement learning without neural nets. I've always avoided neural
> nets--and more generally numerical programming of any sort.
>
> Deep learning has produced so many impressive results that I've decided to
> devote much of my retirement life to learning about it. I retired at the
> end of Spring 2022 and (after a break) am now devoting much of my time to
> learning more about Deep Neural Nets. So far, I've dipped my brain into it
> at various points. I think I've learned a fair amount. For example,
>
>- I now know how to build a neural net (NN) that adds two numbers
>using a single layer with a single neuron. It's really quite simple and is,
>I think, a beautiful example of how NNs work. If I were to teach an intro
>to NNs I'd start with this.
>- I've gone through the Kaggle Deep Learning sequence mentioned
>earlier.
>- I found a paper that shows how you can approximate
>any differentiable function to any degree of accuracy with a single-layer
>NN. (This is a very nice result, although I believe it's not used
>explicitly in building serious Deep NN systems.)
>- From what I've seen so far, most serious DNNs are built using Keras
>rather than PyTorch.
>- I've looked at Jeremy Howard's fast.ai material. I was going to go
>through the course but stopped when I found that it uses PyTorch. Also, it
>seems to be built on fast.ai libraries that do a lot of the work for
>you without explanation.  And it seems to focus almost exclusively on
>Convolutional NNs.
>- My impression of DNNs is that to a great extent they are *ad hoc*.
>There is no good way to determine the best architecture to use for a given
>problem. By architecture, I mean the number of layers, the number of
>neurons in each layer, the types of layers, the activation functions to
>use, etc.
>- All DNNs that I've seen use Python as code glue rather than R or
>some other language. I like Python--so I'm pleased with that.
>- To build serious NNs one should learn the Python libraries Numpy
>(array manipulation) and Pandas (data processing). Numpy especially seems
>to be used for virtually all DNNs that I've seen.
>- Keras and probably PyTorch include a number of special-purpose
>neurons and layers that can be included in one's DNN. These include: a
>DropOut layer, LSTM (short-long-term memory) neurons, convolutional layers,
>recurrent neural net layers (RNN), and more recently transformers, which
>get credit for ChatGPT and related programs. My impression is that these
>special-purpose layers are *ad hoc* in the same sense that functions
>or libraries that one finds useful in a programming language are *ad
>hoc*. They have been very important for the success of DNNs, but they
>came into existence because people invented them in the same way that
>people invented useful functions and libraries.
>- NN libraries also include a menagerie of activation functions. An
>activation function acts as the final control on the output of a layer.
>Different activation functions are used for different purposes. To be
>successful in building a DNN, one must understand what those activation
>functions do for you and which ones to use.
>- I'm especially interested in DNNs that use reinforcement learn