Re: [datameet] Re: 2011 subdistrict Maps
Bihar has one of the most detailed map sets I can see. They have uploaded village boundaries as well. http://gis.bih.nic.in/GISEditor.html Con: works only on InternetExplorer. On Sun, Oct 12, 2014 at 8:04 PM, Khaliq Parkar khaliq.par...@gmail.com wrote: Ah! let me tap my 'institutional' access to see what is possible. will update soon. On 10/12/14, Sharad Lele [शरच्चंद्र लेले] sharad.l...@gmail.com wrote: Khaliq: I have seen similar materials for karnataka and tamil nadu (including also the village amenities directory, which is a mine of information). You are right that if one is to buy this for the whole state, it is quite a bit of money. But the bigger tragedy is (as I had said in a post not so long ago) that Census has, in its 2011 exercise, also created a GIS division that has converted all the boundaries in your pdf file to GIS format, and also gone and mapped down to the level of individual dwellings in many cities. They boast about this in a ppt that is available online which I had circulated, but they do not respond to any requests to make this GIS data base available (for free or for payment). Sharad On 12-Oct-14 11:12 AM, Khaliq Parkar wrote: oh no. apart from the codes, their other data including their not so precise maps. Attaching a 2001 sample. Only Institutional purchase is viable considering the cost. On Sat, Oct 11, 2014 at 11:18 PM, Sharad Lele [शरच्चंद्र लेले] sharad.l...@gmail.com mailto:sharad.l...@gmail.com wrote: Khaliq: When you say they 'sell it', i presume you are referring to the location codes created in each census round and the tables that link one to the other? in which case Sumit Mishra has already posted the all-India dataset for this, right? Sharad On 11-Oct-14 10:19 PM, Khaliq Parkar wrote: Yup, they do. I have all of Maharashtra and all of Bihar, and I can acquire the rest too. I can gather it all and upload it (approx 6-8 GB) if something comes out of it. -- Democratizing Forest Governance in India (In press with Oxford University Press India) -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com mailto:datameet%2bunsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com mailto:datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Democratizing Forest Governance in India (In press with Oxford University Press India) -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [datameet] Re: 2011 subdistrict Maps
No :| But if you don't mind the tedium, select only the boundaries that you want, deselect all basemaps and then you have the transparent outline layer to turn into vector file. The panchayat\village distinction is not clear for *some *Large Villages though. Ignore the error msg on choosing Gram Panchayat. On Mon, Oct 13, 2014 at 3:27 PM, Sharad Lele [शरच्चंद्र लेले] sharad.l...@gmail.com wrote: But this is not downloadable, is it? On 13-Oct-14 2:30 PM, Khaliq Parkar wrote: Bihar has one of the most detailed map sets I can see. They have uploaded village boundaries as well. http://gis.bih.nic.in/GISEditor.html Con: works only on InternetExplorer. On Sun, Oct 12, 2014 at 8:04 PM, Khaliq Parkar khaliq.par...@gmail.com wrote: Ah! let me tap my 'institutional' access to see what is possible. will update soon. On 10/12/14, Sharad Lele [शरच्चंद्र लेले] sharad.l...@gmail.com wrote: Khaliq: I have seen similar materials for karnataka and tamil nadu (including also the village amenities directory, which is a mine of information). You are right that if one is to buy this for the whole state, it is quite a bit of money. But the bigger tragedy is (as I had said in a post not so long ago) that Census has, in its 2011 exercise, also created a GIS division that has converted all the boundaries in your pdf file to GIS format, and also gone and mapped down to the level of individual dwellings in many cities. They boast about this in a ppt that is available online which I had circulated, but they do not respond to any requests to make this GIS data base available (for free or for payment). Sharad On 12-Oct-14 11:12 AM, Khaliq Parkar wrote: oh no. apart from the codes, their other data including their not so precise maps. Attaching a 2001 sample. Only Institutional purchase is viable considering the cost. On Sat, Oct 11, 2014 at 11:18 PM, Sharad Lele [शरच्चंद्र लेले] sharad.l...@gmail.com mailto:sharad.l...@gmail.com wrote: Khaliq: When you say they 'sell it', i presume you are referring to the location codes created in each census round and the tables that link one to the other? in which case Sumit Mishra has already posted the all-India dataset for this, right? Sharad On 11-Oct-14 10:19 PM, Khaliq Parkar wrote: Yup, they do. I have all of Maharashtra and all of Bihar, and I can acquire the rest too. I can gather it all and upload it (approx 6-8 GB) if something comes out of it. -- Democratizing Forest Governance in India (In press with Oxford University Press India) -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe . To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com mailto:datameet%2bunsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com mailto:datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Democratizing Forest Governance in India (In press with Oxford University Press India) -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to a topic in the Google Groups datameet group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/datameet/T6rloStJtBs/unsubscribe. To unsubscribe from this group and all its topics, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Democratizing
Re: [datameet] Re: 2011 subdistrict Maps
As Sharad said, the typologies of the settlement change every census - in fact, the definition of what constitutes a census 'town' for example is determined at the end of current round. e.g. 2001 census determined which settlement would be listed as what *and *how it would be studied in 2011. the census agency has the links for the data, they sell it Yup, they do. I have all of Maharashtra and all of Bihar, and I can acquire the rest too. I can gather it all and upload it (approx 6-8 GB) if something comes out of it. On Wednesday, 1 October 2014 18:32:55 UTC+5:30, Justin Meyers wrote: Sharad, thanks for your help. I am at a point where I have come to accpet the data as is. The 1991, 2001, and 2011 census codes are a great thing, but they will never line up 100%. The data here: https://egovstandards.gov.in/code-dirctory-land-region-codification is missing locations as if they never existed. villages split into new villages, or join larger towns, and the codes are missing. new villages appear, but it doesn't tell you where they came from or how to split existing towns. The http://lgdirectory.gov.in/ website is missing data as well. I have also found out the the census agency has the links for the data, they sell it. But I believe the data they ship has missing data and corrupt files that do not function 100% for some districts/ states. The last little problem with the census data is that it is missing hundreds, if not thousands of locations. In looking at some village level data, there are clearly two or more different villages/ settlements in some of their village polygons. although OSM and wikimapia are crowd projects, I believe some of the data there is correct. Also, if you can find a good district/ tehsil map and it shows many more locations than what the census has, it makes you wonder what is going on... thanks for all of your help with this everyone! Justin On Wednesday, October 1, 2014 8:17:13 AM UTC-4, Sharad Lele wrote: This dataset from Sumit matches what we got from Census. I can add to this the dataset linking 1991 codes to 2001 codes (down to village level), if there is interest. But it will take time. And like Sumit found, I also find some errors here and there: some villages are missed out, or something is wrongly coded. But hard to pinpoint any patterns. Sharad On 14-Sep-14 4:04 PM, Sumit Mishra wrote: Justin/Sharad, Please follow this link to get the files (both in Stata format named - rdir/udir). https://www.dropbox.com/sh/t7d3sy9sage8hoz/AABogZUrrWj-6uyFFy2T8mcna?dl=0 Use this dataset with caution. I had only worked on a subset of this- Maharashtra- and found some coding errors. Let me know if you have any other question. Thanks, Sumit On Sunday, 14 September 2014 00:26:47 UTC+5:30, Justin Meyers wrote: Sharad, Could you send it? Do you have the links to the Web pages? Thanks, Justin On Sep 13, 2014 1:23 PM, Sharad Lele [शरच्चंद्र लेले] shara...@gmail.com wrote: Census, but not on their website. S. On 13-Sep-14 10:27 PM, Justin Meyers wrote: Yes, this is what I am looking for. What is the source? Thanks, Justin On Sep 13, 2014 11:42 AM, Sharad Lele shara...@gmail.com wrote: Are you looking for something like this? Sumit, is this what you have? STC 2001 DTC 2001 Sub-DT 2001 PLCN 2001 NAME OF STATE, DISTRICTS, SUB-DISTTS. VILLAGES 2001 2011 MDDS STC 2011 MDDS DTC 2011 MDDS Sub_DT 2011 MDDS PLCN 2011 MDDS NAME OF STATE, DISTRICT, SUB-DISTTS. VILLAGES 29 00 KARNATAKA 29 000 0 00 KARNATAKA 29 01 Belgaum 29 555 0 00 Belgaum 29 01 0001 Chikodi 29 555 05433 00 Chikodi 29 01 0001 0100 Hadnal 29 555 05433 597114 Hadnal 29 01 0001 0200 Sulagaon 29 555 05433 597115 Sulagaon 29 01 0001 0300 Mattiwade 29 555 05433 597116 Mattiwade 29 01 0001 0400 Bhatnaganur 29 555 05433 597117 Bhatnaganur 29 01 0001 0500 Kurli 29 555 05433 597118 Kurli 29 01 0001 0600 Appachiwadi 29 555 05433 597119 Appachiwadi On Saturday, September 13, 2014 9:39:30 PM UTC+5:30, Justin Meyers wrote: Is it from the ODS website? Could you send it? Thanks, Justin On Sep 13, 2014 1:47 AM, Sumit Mishra sumitm...@gmail.com wrote: I do have one. However, i would forewarn you to use this information with caution; there are too many coding errors. On Saturday, 13 September 2014 00:21:56 UTC+5:30, Justin Meyers wrote: Does anyone have a 2001 subdistrict code that relates to a 2011 subdistrict code table? Thanks! Justin On Thursday, September 11, 2014 11:42:07 PM UTC-4, Justin Meyers wrote: https://lh6.googleusercontent.com/-d2VXra94frA/VBJnm76I9xI/ALw/QQMWlb4rsvA/s1600/indiapolyvillageconvexhull.jpg https://lh4.googleusercontent.com/-VVYOTbkPe-Y/VBJnZGAB7rI/ALo/eU3DFP1pQ7w/s1600/indiapointvillageconvexhull.jpg
[datameet] Re: 2011 subdistrict Maps
As Sharad and Eric have been pointing out, do we have a source for the 2011 subdivision\block boundaries of Indian States? (I plead newbie ignorance if I have missed out something in the vast data flow since the last time I logged in!) On Friday, 12 September 2014 09:12:07 UTC+5:30, Justin Meyers wrote: https://lh6.googleusercontent.com/-d2VXra94frA/VBJnm76I9xI/ALw/QQMWlb4rsvA/s1600/indiapolyvillageconvexhull.jpg https://lh4.googleusercontent.com/-VVYOTbkPe-Y/VBJnZGAB7rI/ALo/eU3DFP1pQ7w/s1600/indiapointvillageconvexhull.jpg https://lh6.googleusercontent.com/-miwMpS2ktlQ/VBJolPkDDsI/AL8/S2_eGmWBrPg/s1600/convex_overlayed_districts2011.jpg putting together couple hundred datasets for india looks like this... based on 2001 data, but i found a translation table to 2011; pretty sure it has a bunch of errors. People who work in the agencies who publish any data really do not respond to e-mails for some mysterious reason... you would think they would want help improving their data (maybe looking good in front of their boss ;) ) but nope!? India is a tough cookie to crack; glad i found everyone on this forum - you have all been beyond kind (especially when I have been somewhat rude). -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups datameet group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [datameet] Re: Shapefiles for complete India
Thanks! Got some from Eric which should suffice, but I hope the 2011 data comes through. On Saturday, 11 October 2014 11:19:26 UTC+5:30, Sharad Lele wrote: Khaliq, I guess after Justin's recent email, he should be in the best position to give you the block boundary shapefiles for Bihar. Sharad On Friday, October 10, 2014 7:20:04 AM UTC+5:30, Khaliq Parkar wrote: Hi Eric, I am working in Bihar within Madhubani district. Could you link me to the block level shapefiles? Do you know anyone who has worked at village\panchayat level divisions? Thanks! Khaliq On Thursday, 7 August 2014 20:15:45 UTC+5:30, Eric Dodge wrote: This is very interesting Sharad. I've been looking for maps of what I've been calling administrative blocks, that is, the units overseen by block development officers. MGNREGA data is aggregated at this level and I've been hoping to use the data to do some mapping exercises. The census sub-districts are called differently across states (tahsil, taluk, mandal, etc). You can see the list here: http://censusindia.gov.in/Tables_Published/Admin_Units/Admin_links/subdistrict_nomeclature.html I know that in all the states where census sub-districts are called taluk, mandal, or CD block (with the exception of TN), the census sub-district is identical to the administrative block. I've already completed a mapping exercise for Bihar using the census sub-district map and the data matched up pretty well. If the IND_adm3 data is indeed the administrative blocks then I could do a similar exercise with Madhya Pradesh. I'll take a look to see if the data lines up correctly. Has anybody dug into this issue any deeper? I've heard that tehsil comes from the revenue side whereas taluk, mandal, etc comes from the administrative side but that doesn't explain why the census uses different sub-district units across states. Best, Eric On Thu, Aug 7, 2014 at 7:39 AM, Sharad Lele shara...@gmail.com wrote: If I am right, then Justin may want to rename his layer as CDBlocks_2001... Sharad On Thursday, August 7, 2014 4:28:17 PM UTC+5:30, Sharad Lele wrote: I think I have the explanation for why I am seeing a good match and you are not: The problem lies in defining what is the 'sub-district' unit (in IND_adm3). Administratively speaking, it is tehsil, below which lies CD block. Unfortunately, census gives information by CD block. So there are more 'sub-district' units in Census than tehsils in the country. GDAM seems to have followed the tehsil concept. To check: Karnataka is one state in which tehsil and CD block are one and the same. That is why the sub-district layer IND_adm3 matches perfectly for Karnataka, but not for other states. There might be some other states where this holds good, I don't know. Anyway, so if one really wants CD block level boundaries, we have to look at Justin, I guess. But the GDAM boundaries are not 'wrong'. Sharad On 07-Aug-14 9:48 AM, Devdatta Tengshe wrote: In Continuation of my previous email, here is a CSV file which shows just how bad the GDAM dataset is. Regards, Devdatta On Thu, Aug 7, 2014 at 9:06 AM, Devdatta Tengshe devd...@tengshe.in wrote: Hi Sharad, I just download the GDAM data again, to confirm what you have said. I'm going to have to disagree with you about the quality of the IND_adm3 data. Acoording to the 2001 Census, there are 5454 Sub Districts in India. The GDAM dataset has just 2299 features. So clearly these taluk features do not correspond to the 2001 Census. I cross checked for some areas I have ground knowledge of, and I can say that this dataset is not from any specific era. Some tehsils in the file were created post 2001, while others created in the 90's were not present. In my opinion the GDAM data is pretty much unusable. Regards, Devdatta
Re: [datameet] Re: Shapefiles for complete India
Hi Eric, I am working in Bihar within Madhubani district. Could you link me to the block level shapefiles? Do you know anyone who has worked at village\panchayat level divisions? Thanks! Khaliq On Thursday, 7 August 2014 20:15:45 UTC+5:30, Eric Dodge wrote: This is very interesting Sharad. I've been looking for maps of what I've been calling administrative blocks, that is, the units overseen by block development officers. MGNREGA data is aggregated at this level and I've been hoping to use the data to do some mapping exercises. The census sub-districts are called differently across states (tahsil, taluk, mandal, etc). You can see the list here: http://censusindia.gov.in/Tables_Published/Admin_Units/Admin_links/subdistrict_nomeclature.html I know that in all the states where census sub-districts are called taluk, mandal, or CD block (with the exception of TN), the census sub-district is identical to the administrative block. I've already completed a mapping exercise for Bihar using the census sub-district map and the data matched up pretty well. If the IND_adm3 data is indeed the administrative blocks then I could do a similar exercise with Madhya Pradesh. I'll take a look to see if the data lines up correctly. Has anybody dug into this issue any deeper? I've heard that tehsil comes from the revenue side whereas taluk, mandal, etc comes from the administrative side but that doesn't explain why the census uses different sub-district units across states. Best, Eric On Thu, Aug 7, 2014 at 7:39 AM, Sharad Lele shara...@gmail.com wrote: If I am right, then Justin may want to rename his layer as CDBlocks_2001... Sharad On Thursday, August 7, 2014 4:28:17 PM UTC+5:30, Sharad Lele wrote: I think I have the explanation for why I am seeing a good match and you are not: The problem lies in defining what is the 'sub-district' unit (in IND_adm3). Administratively speaking, it is tehsil, below which lies CD block. Unfortunately, census gives information by CD block. So there are more 'sub-district' units in Census than tehsils in the country. GDAM seems to have followed the tehsil concept. To check: Karnataka is one state in which tehsil and CD block are one and the same. That is why the sub-district layer IND_adm3 matches perfectly for Karnataka, but not for other states. There might be some other states where this holds good, I don't know. Anyway, so if one really wants CD block level boundaries, we have to look at Justin, I guess. But the GDAM boundaries are not 'wrong'. Sharad On 07-Aug-14 9:48 AM, Devdatta Tengshe wrote: In Continuation of my previous email, here is a CSV file which shows just how bad the GDAM dataset is. Regards, Devdatta On Thu, Aug 7, 2014 at 9:06 AM, Devdatta Tengshe devd...@tengshe.in wrote: Hi Sharad, I just download the GDAM data again, to confirm what you have said. I'm going to have to disagree with you about the quality of the IND_adm3 data. Acoording to the 2001 Census, there are 5454 Sub Districts in India. The GDAM dataset has just 2299 features. So clearly these taluk features do not correspond to the 2001 Census. I cross checked for some areas I have ground knowledge of, and I can say that this dataset is not from any specific era. Some tehsils in the file were created post 2001, while others created in the 90's were not present. In my opinion the GDAM data is pretty much unusable. Regards, Devdatta On Wed, Aug 6, 2014 at 11:36 PM, Sharad Lele shara...@gmail.com wrote: I have downloaded and checked the GADM boundaries (my version is 2011). The taluka boundary layer probably holds good today, becuase few talukas get split. Districts get split regularly (every so many years) so the district boundary layer in this GADM set is quite
