Hi data enthusiasts
I have a lot of publicly available data which are pdf scans of old
publications. I wish to digitize them as a public service. I found that the
following python package is pretty efficient at doing this job:
https://layout-parser.readthedocs.io/en/latest/
However, since I am
Hi Ravi
I'm not entirely aware of a bank branch data set that comes with lat-long
info. However, the SHRUG data thatI alluded to includes PIN codes of
branches. So you can merge that data with the following PIN code map to get
approximate locations of bank branches:
Hi Goutham
I tried accessing the archived versions of the older input survey website
on web.archive.org (a website that automatically and randomly archives
webpage over the internet) thinking that maybe one could get access to the
data from there. But unfortunately, the data on that old website
Hi Ravi
https://www.devdatalab.org/shrug_download/
On this webpage, search for the term 'bank' and you will get the data. This
data is at the bank branch level by village. The village names etc are
available in the 'core keys' file.
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
Hi Neha
To the best of my knowledge, the only household survey that is conducted
annually is the Consumer Pyramids Survey by the Centre for Monitoring
Indian Economy. But there are two caveats. One, I highly doubt that it
consists of the variable you are looking for. Two, it can be accessed only
Try the shapefiles from NSS 1993 here:
https://www.dropbox.com/scl/fo/xgbc95022gqs9hg66tauk/h?rlkey=qbd4hza0vw21xutju3intrjxy=0
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State University of New York)
Hey Nikhil
Thanks for correcting me!
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State University of New York)
Zoom ID: 201 717 2613
www.shivhastawala.com
On Tue, 13 Feb 2024 at 1:43 pm, Nikhil VJ
Hi Sudarshan
Disclaimer: I haven't tried it and I'm only shooting in the dark.
Open street map (OSM) usually has archives of a lot of types of their data
across the web. It also has a GPS navigation system similar to Google maps,
so I'm assuming they would have archival data for traffic too.
My pleasure!
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State University of New York)
Zoom ID: 201 717 2613
www.shivhastawala.com
On Fri, 2 Feb 2024 at 3:28 am, Nadine K.J. wrote:
> Thanks Shiv!!
>
>
Actually all place names are available in the file 'shrid_loc_names'.
Please go through their documentation to understand how they define a
SHRID. It is basically a village, but you will get the nuances from their
docs.
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Hi Nadine
You can find that in Table A01 of both the 2001 and 2011 Censuses. You can
access those tables here:
https://censusindia.gov.in/census.website/data/census-tables#
You will need to select the census year on the left to get the tables from
that year.
Thanks and regards.
Yours
niversity
>
>
> On Tue, Jan 30, 2024 at 12:27 AM 'Shiv Hastawala' via datameet <
> datameet@googlegroups.com> wrote:
>
>> Hi Vaibhav
>>
>> Data on bureaucrats has already been compiled by the Trivedi Centre for
>> Political Data here:
>>
>>
&
Jan 2024 at 5:29 am, Anand Katakam wrote:
> Thanks so much, Shiv. If I was to visualise this data, how would I
> attribute it? To the ECI or to you?
>
>
> Anand Katakam
> an...@katakam.com
> www.anand.katakam.com
>
>
>
> On Wed, 31 Jan 2024 at 15:33, 'Shiv Has
Hi Anand
The data is available here (downloaded from different github repos):
https://www.dropbox.com/sh/c0r2kks1pmzoe13/AABudfkgMIOrg3iOJWXIY2ZIa?dl=0
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State
Hi Sudarshan
I haven't tried it out, but there are some guidelines given here:
https://opendata.stackexchange.com/questions/18330/seeking-locations-data-of-all-police-stations-in-india
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of
Hi Vaibhav
Data on bureaucrats has already been compiled by the Trivedi Centre for
Political Data here:
https://web.archive.org/web/20220520194317/https://github.com/tcpd/ias-release
Is this what you were looking for?
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Hi Nadine
It is available here:
https://www.devdatalab.org/shrug_download/
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State University of New York)
Zoom ID: 201 717 2613
www.shivhastawala.com
On
Hi Sagar
The Population Census module of SHRUG 2.0 consists of village-level caste
data from the 1991, 2001, and 2011 Censuses:
https://www.devdatalab.org/shrug_download/
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton
Hi Rikhia
You can select multiple filters on the following link to find census data
at the district level (my guess is you should be able to find what you are
looking for). To begin with, try selecting 2001 and then Education:
https://censusindia.gov.in/census.website/data/census-tables
uld be atleast ~440 of these
> and I'm wondering if any other website hosts the remaining districts.
>
> On Thu, 5 Oct 2023, 16:14 'Shiv Hastawala' via datameet, <
> data...@googlegroups.com> wrote:
>
>> About 130 of them show up here:
>>
>>
>> https://c
Hi Sharad
I'm sorry, I haven't ever worked with shapefiles or geopkg files so I'm
afraid I don't have expertise in this area. I just had access to these data
files, so I passed them on.
On Wed, 6 Dec 2023 at 8:19 am, Sharad Lele wrote:
> Hi Nikhil, Ujaval, and others who responded:
> Shiv, my
d/soi-india-map-toponyms>
>
> Regards,
> Devdatta
>
>
> On Mon, 4 Dec 2023 at 02:51, 'Shiv Hastawala' via datameet <
> datameet@googlegroups.com> wrote:
>
>> Hi Ravi
>>
>> What you are looking for should be available in the folder titled *"13.
>>
Hi Ravi
What you are looking for should be available in the folder titled *"13.
OPEN POLYGONS AND SPATIAL STATISTICS"* from SHRUG 2.0 here:
https://www.dropbox.com/scl/fo/it0dk1znotiap9w2zjz7l/h?rlkey=qjsueenrj7xrg2v9wsob5iyt5=0
Don't forget to match the Shrid IDs from the folder titled *"1.
Hi Mantha
There isn't a readily-available crosswalk, but there might just be a
workaround. I can give you some resources but you would have to be
*extremely* careful while matching stuff across these data sets because
district, AC, and PIN code *boundaries are very fluid* and keep changing.
You
Hi Parvati
You can find all unit-level MOSPI data sets here on the National Data
Archive:
https://microdata.gov.in/nada43/index.php/catalog
Click on 'Download Manual' to understand the procedure to get access to all
data sets. You would have to register as a user on the portal, and then you
Nope, I haven't cleaned this data set in any manner. Sorry about that!
On Thu, 30 Nov 2023 at 1:42 pm, rc wrote:
> Hi Shiv, Thanks for sharing. But to use this geometry lot of topology and
> manual changes would be required. By any chance do you have cleaned
> boundary dataset?
>
> On Tuesday,
My pleasure!
On Tue, 28 Nov 2023 at 1:06 pm, Digvijay Bendrikar Shinde <
digvijaybendrikarshi...@gmail.com> wrote:
> Wow this is Amazing resource Shiv!!
>
> Thank you very much!!!
>
> Digvijay
>
> On Tue, 28 Nov, 2023, 11:14 pm 'Shiv Hastawala' via datameet, <
>
Hi Ravi
Please visit my website below, and select the tab named 'Geographic' on the
Google Sheet. You would be able to see information regarding all boundary
maps of India and links to their repositiries, including Parliamentary and
Assembly constituencies.
https://www.shivhastawala.com/data
Institute for Social & Economic Change, *
> *Bengaluru - 560072*
> *Email- kpmishra1...@gmail.com *
> *kaibalyap...@isec.ac.in *
>
>
> On Mon, Nov 27, 2023 at 2:00 AM 'Shiv Hastawala' via datameet <
> datameet@googlegroups.com> wrote:
>
>> H
Hi Kaibalya
You can find DISE school-level data for 2005-2018 here:
https://www.dropbox.com/scl/fo/o3hes1ixfv3zhgrwung5x/h?rlkey=mnlzddqgf4qqvs22whst4sk6u=0
Also, you can download data beyond 2018 from here by signing up and logging
in:
https://src.udiseplus.gov.in/udise-share/dashboard
On
Hi Vaibhav
I'm not aware of a readily available data set that has asset breakup as
mentioned by you. However, I remember one of the repositories that I
provided the link for before consists of the code used for scraping this
data. Maybe you can modify that to suit your needs?
On Sat, Nov 11,
Hi Naresh
The latest Census data isn't available (I'm not sure if it has even been
conducted) but 2021 population and wealth estimates are available at the
village level in the folder called 'Facebook Wealth and Population 2021' on
this link (please read the Metadata to understand how these
Hi Suraj
You can find elevation data for India in one of the folders on this link
(please go over the first folder to match SHRUG IDs in the elevation
folder):
https://www.dropbox.com/scl/fo/it0dk1znotiap9w2zjz7l/h?rlkey=qjsueenrj7xrg2v9wsob5iyt5=0
On Tue, Nov 14, 2023 at 7:29 AM, Shrobona
Hi Rachit
I haven't used this data set so I'm not entirely aware of how to use it.
But the following Git repos may be of some use to you:
https://github.com/gantian127/bmi_era5
https://github.com/gmanatole/API-ERA5
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral
Hi Vaibhav
The following link contains data for 2000-2021 Vidhan Sabha candidates:
https://www.dropbox.com/scl/fo/nau1pxt5c4zmc7vz13f3c/h?rlkey=0o766w24fg0vrt4zncbu92r9i=0
The following folder contains data for 1961-2015 Vidhan Sabha candidates:
Hi Himanshu
Not sure if this is what you need, but maybe try it:
https://www.dropbox.com/sh/kpn8rfltv3c59v5/AABxq_cNODliJeNDJ6QN578pa?dl=0
On Tue, Oct 31, 2023 at 6:05 PM, Himanshu wrote:
> I am unable to find Delhi NCR boundary shape file. Any leads?
>
> --
> Datameet is a community of Data
I was able to download all the DISE 2005-2018 data from the link I posted
before. The data is available here:
https://www.dropbox.com/scl/fo/o3hes1ixfv3zhgrwung5x/h?rlkey=mnlzddqgf4qqvs22whst4sk6u=0
On Thursday, July 6, 2023 at 3:10:42 AM UTC-4 Shiv Hastawala wrote:
> Hi Neha
>
> I used to be
Available (only up to 2013) here:
http://data.icrisat.org/dld/src/gdp.html
https://data.gov.in/catalog/district-wise-gdp-and-growth-rate-constant-price2004-05
https://data.gov.in/ministrydepartment/NITI%20Aayog
On Fri, Oct 20, 2023 at 8:55 AM, Devdatta Tengshe
wrote:
> I too was searching
About 130 of them show up here:
https://censusindia.gov.in/nada/index.php/catalog/?page=1=district%20census%20handbook%201991
On Thu, Oct 5, 2023 at 6:32 AM, Kritarth Jha wrote:
> Hi All,
>
> Does anyone know where to find the digitised district handbooks for the
> 1991 census? I know it used
Hi Golam
This is another data set with district-level night light data between
1992-2021 from DMSP and VIIRS: https://www.devdatalab.org/shrug_download/
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State
Actually, this seems to be a much more stable collection of subdistrict
shapefiles:
https://www.devdatalab.org/shrug_download/
On Tue, Oct 3, 2023 at 3:44 AM, Shiv Hastawala
wrote:
> Hi Patrick
>
> Try this:
>
> https://wiki.openstreetmap.org/wiki/Subdistricts_in_India
>
>
> I got the idea
Hi Patrick
Try this:
https://wiki.openstreetmap.org/wiki/Subdistricts_in_India
I got the idea from this paper here:
Try this:
https://github.com/HindustanTimesLabs/shapefiles/tree/master/india/city
On Sat, Sep 23, 2023 at 12:04 AM, rc wrote:
> Hello everyone,
> I am looking for administrative boundaries for cities and towns in India.
> Any leads will be super helpful.
>
> Thanks
> Ravi
>
> --
> Datameet is
Hi Golam
District level GDP per se is available for only 2007-2013 in the following
data set:
https://www.dropbox.com/sh/cbghovzdldah0rg/AACgmhUltFWDDoXOQ9VrDy5ga?dl=0
However, economists have been using night light data as a very good proxy
for GDP at levels like the district (because GDP data
Hi Neha
I used to be able to download all school-level data for years before 2018
from the following link without the requirement for logging in but the link
doesnt seem to be working anymore (maybe it's a location thing since I am
outside India; it might work for you if one tries to access it
Ah yes, my bad. I'm not aware of usage of MLA funds. Sorry for the
confusion.
On Thu, Jun 29, 2023 at 10:25 PM, BBCnews india
wrote:
> Thank you Shiv but the dataset is for the MPLADS not for the MLA funds.
>
> On Thu, 29 Jun 2023 at 05:24, 'Shiv Hastawala' via datameet <
Try this:
https://www.dropbox.com/sh/v1rbe8i43jlonk5/AADrS1mfpqhoMGyJIP2wiWHia?dl%3D0=D=editors=168755755249=AOvVaw0CRsKlw2wM6GSz3ndl10ta
On Wed, Jun 28, 2023 at 7:30 PM, BBCnews india
wrote:
> Respected members
>
> I hope this email finds you in good health and high spirits.
>
> I am
Hi Yasir
I'm not sure about a district crosswalk specifically for the NSS, but I
have a couple of district crosswalks available below that you should be
able to match with NSS districts:
https://www.dropbox.com/sh/uomplqrqrb554qo/AADEzcCYuX0ZD2kxw9_kD6BZa?dl=0
Hi Gunngunn
You can head over to this page on my website and scroll over to the
'Socioeconomic' tab at the bottom of the spreadsheet. There you will find a
list of all useful datasets for socioeconomic variables, including the
latest ones. One data set that is not on this tab but under the
*NCO codes
On Thu, Jun 1, 2023 at 6:54 AM, Shiv Hastawala
wrote:
> Hi Bilal
>
> Last I had checked, the maximum most of the Indian surveys collected was 3
> digit NOC codes. I haven't been able to find a survey that collects data on
> more than 3 digit codes.
>
> On Wed, May 31, 2023 at 1:31
Hi Bilal
Last I had checked, the maximum most of the Indian surveys collected was 3
digit NOC codes. I haven't been able to find a survey that collects data on
more than 3 digit codes.
On Wed, May 31, 2023 at 1:31 PM, Bilal Moin
wrote:
> Does any economic census, population census, or NSS
You're welcome!
On Wed, May 31, 2023 at 1:18 PM, VASTAV RATRA <
f20180...@hyderabad.bits-pilani.ac.in> wrote:
> Thanks a lot, Shiv. I'll try contacting Prof. Foster to get access to REDS
> 2006 for my project. This was really helpful!
>
> On Wednesday, May 24, 2023 at 3:29:07 AM UTC+5:30 Shiv
Hi Vastav
There is a household level survey carried out by the NCAER since 1982 known
as the Rural Economic and Demographic Survey (REDS) which includes a lot of
variables on livestock. There are 3 waves of the data set in which I could
find these variables: 1982, 1999, and 2006. The latest 2014
Hi Anisha
I don;t think the SHRUG data set can be broken down any further in terms of
geographic boundaries. What you are looking for is not readily available
but you can create that data for yourself using vector maps of two things:
granular night light data and ward boundaries for Delhi. But
Hi Shyamal
This Datameet repo might be of some use for your project:
https://github.com/datameet/railways
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University (State University of New York)
Zoom ID: 201 717 2613
Hi Till
This might just be what you need (please read the paper in the folder to
get context of the data set):
https://www.dropbox.com/sh/rt0tldkybbdccmb/AAChviOyKfqZo3zPpe84xGyHa?dl=0
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Hi Sayali
There isn't a dataset which has exactly what you need but you can merge the
following two data sets and create it for yourself (merge by district)
District-AC crosswalk (Please read the paper in the folder to get context):
ntre for Economic Studies and Policy,
> Institute for Social and Economic Change, Bangalore
> https://kaibalyapatimishra.godaddysites.com/
>
>
> On Wed, May 3, 2023 at 6:21 AM 'Shiv Hastawala' via datameet <
> datameet@googlegroups.com> wrote:
>
>> Sorry f
Sorry for the incorrect link in the previous message.
Here is the link to the repository:
https://www.dropbox.com/sh/1fei5xrz0e4so76/AAAxpHJgGFeXht-6zo7fBc9Na?dl=0
Thanks and regards.
Yours sincerely
*Shiv Hastawala*
(He/His/Him)
Doctoral Student
Department of Economics
Binghamton University
Hi Kaibalya
For employment and unemployment data you can see the link that Chris has
posted on this thread.
For PLFS, you can see my repository here:
Nope, we don't have data that is that detailed. However, there is data
collected by PRS Legislative regarding MP/MLAs' attendance, number of
questions asked in Parliament/Vidhan Sabha etc. It spans about 3-4
variables. I'm not sure if you already know about it, but you can access
the data here:
Hi Ryu
I'm not sure about what you mean by 'how an MP/MLA cast a ballot in their
legislation'. Casting a ballot usually refers to the act of voting in an
election. If you mean to say that you need info on who these politicians
voted for (which they do since they are also citizens of the nation)
Another one is the COVID19 Related Shocks in Rural India Survey conducted
in 3 different rounds.
Here are the links to the datasets:
https://microdata.worldbank.org/index.php/catalog/3769
https://microdata.worldbank.org/index.php/catalog/3773
Hi Nikhil
This might be relevant: the Covid 19 Livelihoods Survey by Azim Premji
University
https://www.dropbox.com/sh/8z6o8ra1duouokb/AADaAiNkNJtB5AAaHhTJ-6exa?dl=0
On Sat, Mar 18, 2023 at 2:56 AM, Nikhil VJ wrote:
> Hi, can anyone connect me to any kind of survey / stats data on the socio
Hi Vaidya
Maybe not maps, but the following data sets have lat-long data for polling
booths, please check them out:
https://www.dropbox.com/sh/c0r2kks1pmzoe13/AABudfkgMIOrg3iOJWXIY2ZIa?dl=0
On Mon, Dec 26, 2022 at 11:10 AM, Vaidya wrote:
> Hello,
>
> Does anyone have Delhi's polling booth
Hi Sahil
Although this data doesn't seem to be readily available, there are several
projects on GitHub with the help of which you can scrape the data you want.
Here are some examples:
https://github.com/Skumarr53/Stock-Fundamental-data-scraping-and-analysis
Hi Purnima and Ashwin
I have gathered information on several Indian political data sets on my
website: https://www.shivhastawala.com/data
You need to navigate to the tab at the bottom of the spreadsheet titled
'Political/Democratic/Civic'. I hope you are able to find some relevant
stuff there.
67 matches
Mail list logo
