data science tutorials

[AudioGames . net Forum — Off-topic room : visualstudio via Audiogames-reflector]

data science tutorials

hi,
some days ago if you remember, I've posted a topic and asked about making tutorials for data science and the answers of all of you were yes.
now, I'm going to do that
@mods, feel free to move the topic if here is not it's place (I've created this topic based on the suggestion to make it here in the off topic)
also if you have any questions, feel free to ask.

prerequisites

• experience in python (I won't teach python)

• some math skills like linear algibra or probobility (though I teach them, but having this skill will help you more)

• a computer

packages that we need during the tutorials

you don't need to install all of these, since we won't use all of them in all of the tutorials.
but, I will write them here if you want to install them.

• numpy

• scipy

• pandas

• scikit-learn

• opencv-python

• spacy

• torch

• torchvision

• librosa

• jupytext

numpy

to do neumerical computation with arrays, we need numpy
in order to install it, we do the following in the command prompt:

pip install numpy

scipy

for scientific computation, we need scipy
it has some functionality like loading wav files into numpy arrays, spars arrays etc.
to install it, like numpy we do it from the command line like this:

pip install scipy

pandas

this one is different from numpy and scipy, and it is used for loading the data and preprocessing them
consider that you have a tabular data (more on that later), and you want to load it and preprocess it before feeding it to your ml algorithm
it has many many more functionality than python's csv module, and it can load a variety of types.
to install it, we use pip like the following:

pip install pandas

scikit-learn

this is a library for machine learning
it has many algorithms, which we will cover some of them
it also has many algorithms for preprocessing data, many algorithms for calculating how accurate our model is, etc.
to install it we do the following like other libraries, with the difference that we install scikit-learn instead of the previous ones
the command is as follows:

pip install scikit-learn

opencv-python

this one is used to load images and many many algorithms to process them
it's name is opensource computer vision, so it needs to have these algorithms
for example, you can load an image, convert it into grayscale, threshold it, do edge detection etc on them (which I will cover them).
to install it:

pip install opencv-python

spacy

have you ever wandered how can we do natural language processing (nlp)?
oh. what is nlp?
nlp or natural language processing is processing linguistic data
it will fall into 2 categories:

• nlu or natural language understanding which is used to extract data from the language

• natural language generation which is used to generate sentences or phrases similar to humans

now, spacy will help us in nlp
to install it,

pip install spacy

also, we have another step here, sorry

python -m spacy download en

this will download small dataset for english (instead of en you can use en_core_web_sm or en_core_web_lg which will download midium or large datasets respectively)

torch and torchvision

this is a library for neural networks, which has support for running on the gpu
torchvision is for load and/or preprocessing images
you can go here and select your requirements and it will give you a command to install them.
but for simplicity, I will give the command to install them for cpu, which is as follows:

pip install torch==1.4.0+cpu torchvision==0.5.0+cpu -f https://download.pytorch.org/whl/torch_stable.html

to install it for gpu, you need to download and install cuda and cudnn first.

librosa

like opencv-python which was used to load images, this one is used to load and process audio
the command to install it is:

pip install librosa

jupytext

unfortunately, we can't write our code in jupyter notebook or jupyter lab
so, we will write them in normal python files, and this application will turn it into notebooks which we can run in google colab or the jupyter notebook if we installed it in our computer.
the command to install it is as follows:

pip install jupytext

what is google colab?

google colab gives you free access to a jupyter with  gpu for training your ml models.
so, if you don't have a nvidia gpu or don't have cuda and cudnn installed, colab is your best bet
also many of the packages like torch, opencv-python are preinstalled on it.

URL: https://forum.audiogames.net/post/503757/#p503757

-- 
Audiogames-reflector mailing list
Audiogames-reflector@sabahattin-gucukoglu.com
https://sabahattin-gucukoglu.com/cgi-bin/mailman/listinfo/audiogames-reflector