Re: Ram memory not freed after executing python script on ubuntu system

2020-05-28 Thread Rahul Gupta
On Thursday, May 28, 2020 at 11:20:05 AM UTC+5:30, Rahul Gupta wrote: > I am having a Ubuntu system which has 125 Gb of RAM. I executed few python > scripts on that system. Those scripts uses numpy arrays and pandas. Now > execution was over but still 50 gb of RAM and 2 Gb cache and

Ram memory not freed after executing python script on ubuntu system

2020-05-27 Thread Rahul Gupta
I am having a Ubuntu system which has 125 Gb of RAM. I executed few python scripts on that system. Those scripts uses numpy arrays and pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is occupied. At this moment nothing is running on the system. I have go

Re: error in CSV resetting with seek(0)

2020-05-02 Thread Rahul Gupta
@peter Otten thanks -- https://mail.python.org/mailman/listinfo/python-list

error in CSV resetting with seek(0)

2020-05-01 Thread Rahul Gupta
consider the following code import csv import numpy as np with open("D:\PHD\obranking\\demo.csv", mode='r') as csv_file1, open("D:\PHD\obranking\\demo.csv", mode='r') as csv_file2: csv_reader1 = csv.DictReader(csv_file1) csv_reader2 = csv.DictReader(csv_file2) filename = "cell_split

unable to write content in csv filw

2020-04-27 Thread Rahul Gupta
FOLLWOING IS MY CODE import pandas as pd import csv from sklearn.preprocessing import LabelEncoder from sklearn.feature_selection import chi2 with open("D:\PHD\obranking\\test_chi.csv", 'w') as csvfilew1: fields = ['index', 'feature name', 'p_value'] csvwriter1 = csv.DictWriter(csvfilew1, f

chi square test in sklearn printing NAN values for most of the columns

2020-04-27 Thread Rahul Gupta
chi value for 10th column but for most of the columns it is behaving like below "C:\Users\Rahul Gupta\PycharmProjects\CSVLearn\venv\Scripts\python.exe" "C:/Users/Rahul Gupta/PycharmProjects/CSVLearn/ChiSq_learn.py" (array([nan]), array([nan])) Process finished with exit c

Incremental PCA

2020-04-18 Thread Rahul Gupta
i wanted to implement incremental PCA. Got this code for stack overflow but i am wondering what y = chunk.pop("y") does and what is this argument "y" to pop from sklearn.decomposition import IncrementalPCA import csv import sys import numpy as np import pandas as pd dataset = sys.argv[1] chunksiz

Re: To apply pca for a large csv

2020-04-14 Thread Rahul Gupta
64 bit version -- https://mail.python.org/mailman/listinfo/python-list

To apply pca for a large csv

2020-04-14 Thread Rahul Gupta
Hello all, i have a csv of 1 gb which consists of 25000 columns and 2 rows. I want to apply pca so i have seen sciki-learn had inbuilt fucntionality to use that. But i have seen to do eo you have to load data in data frame. But my machine is i5 with 8 gb of ram which fails to load all this d

Re: i am want to read data from the csv that i wrote using python csv module but apart from filed names and row count i am unable to read rest of the data

2020-04-12 Thread Rahul Gupta
@Peter Thanks alot -- https://mail.python.org/mailman/listinfo/python-list

Re: i am want to read data from the csv that i wrote using python csv module but apart from filed names and row count i am unable to read rest of the data

2020-04-12 Thread Rahul Gupta
import csv import numpy as np with open("D:\PHD\obranking\\cell_split_demo.csv", mode='r') as csv_file: csv_reader = csv.DictReader(csv_file) print(csv_reader.fieldnames) col_count = print(len(csv_reader.fieldnames)) #print(sum(1 for row in csv_file)) row_count = 0

Re: i am want to read data from the csv that i wrote using python csv module but apart from filed names and row count i am unable to read rest of the data

2020-04-12 Thread Rahul Gupta
On Sunday, April 12, 2020 at 1:35:10 PM UTC+5:30, Rahul Gupta wrote: > the cells in my csv that i wrote looks likes this > ['82#201#426#553#602#621#811#908#1289#1342#1401#1472#1593#1641#1794#2290#2341#2391#3023#3141#3227#3240#3525#3529#3690#3881#4406#4421#4497#4719#4722#4920#505

i am want to read data from the csv that i wrote using python csv module but apart from filed names and row count i am unable to read rest of the data

2020-04-12 Thread Rahul Gupta
the cells in my csv that i wrote looks likes this ['82#201#426#553#602#621#811#908#1289#1342#1401#1472#1593#1641#1794#2290#2341#2391#3023#3141#3227#3240#3525#3529#3690#3881#4406#4421#4497#4719#4722#4920#5053#5146#5433'] and the cells which are empty looks like [''] i have tried the following code