Hi Bryan All the time stamps are Indian Standard Time.
Rahul Today is the first day of the rest of your life ! On Sat, Sep 5, 2026 at 8:28 AM Rahul Basu <[email protected]> wrote: > Wow. That's amazing. And so quick.Thanks a ton Bryan. > > Rahul > Today is the first day of the rest of your life ! > > > On Sat, Sep 5, 2026 at 3:43 AM Bryan <[email protected]> wrote: > >> Hi Rahul and everyone, >> >> I've converted the LAQ 167 archive into a SQLite database, with >> compressed JSON records and smaller JSON tables alongside it: >> >> https://drive.google.com/file/d/1Npw5WK7eOCr7hKzRiirjT0lgBGLDRcKS/view >> >> The ZIP is about 31 MB. Download and extract it, then open >> goa-mining.sqlite in a SQLite viewer and select the trip_evidence view. The >> README explains the tables, and QUERIES.sql includes starting queries. This >> is a free community contribution, with no paid service needed to use the >> file. >> >> What's in it: >> - 363,507 individual trips from all ten Annexure D PDFs, covering 55,420 >> pages. >> - All 798 published daily route totals, plus block/lessee, monthly >> production, dispatch and route-permission context from A/B/C. >> - Source PDF names, page references and hashes, so a record can be traced >> back to the original document. >> >> The trip-level counts and ore quantities reconcile exactly with all 798 >> published daily route totals. There are no duplicate trip IDs or parsing >> exceptions. I also checked every cell in 290 rows across 36 sampled pages >> with a second PDF parser; that independent check is sampled, not every page. >> >> The actual trip dates range from 6 July 2024 to 19 August 2026, with >> different cutoffs by block. The September answer date is not a claim that >> every block is current to September. Block II has contextual records but no >> trip PDF in Annexure D. >> >> All trips match a published route-permission entry using the route text, >> without fuzzy matching. The package also flags a 0.01 difference in one >> stated production total and four differing commencement-date statements >> across annexures, retaining both versions. The printed clocks lack >> AM/PM/timezone, so I haven't turned them into precise timestamps or claimed >> hourly violations. AQI remains a linked companion source, not an inferred >> route-to-sensor or causal match. >> >> If a row looks wrong, a PDF/page reference or trip ID will let us check >> it precisely. I hope this saves the group a substantial amount of >> preparation work. >> >> Bryan >> CivicDataForge >> [email protected] >> On Friday, September 4, 2026 at 3:03:12 AM UTC-4 Rahul Basu wrote: >> >>> On mining transport, there's a big set of data as part of the >>> Legislative Assembly Questions. Basically all the trips for major mining >>> leases, routes, etc. It is in multiple PDFs, would greatly appreciate it if >>> anyone can process these into databases. >>> http://static.goavidhansabha.gov.in/goalpub/vr/167_01092026.zip >>> >>> Thanks in advance. >>> >>> Rahul >>> Today is the first day of the rest of your life ! >>> >>> >>> On Wed, Sep 2, 2026 at 2:56 PM Rahul Basu <[email protected]> wrote: >>> >>>> Friends, esp Sanjay & Bryan, >>>> >>>> I've organised the data better. There's a bunch of sub-folders in this >>>> folder >>>> <https://drive.google.com/drive/folders/1crvN-kiKZUYgFeu7PZwsjyhcYc99Q4ZA?usp=sharing>. >>>> In each folder, I've tried to create a doc file that provides some >>>> background. >>>> >>>> The mining transport data that Sanjay worked on is in Mining Transport >>>> - Pub >>>> <https://drive.google.com/drive/folders/1GTurKNqV02XI3_79zlDpQXZhf9w_C2bR?usp=sharing>. >>>> I've added some older data to this folder, as well as newer data (could be >>>> some overlap in periods). >>>> >>>> BTW, the work around the AQI data logger >>>> <https://groups.google.com/g/datameet/c/k5H0xQqwSc4/m/AUD3QsYnDAAJ> by >>>> Arun is basically air quality data from stations established specifically >>>> to monitor air quality along these transportation routes. >>>> >>>> Please point out any additional information that may be useful for >>>> analysis. >>>> >>>> Looking forward to the insights of the community. >>>> >>>> Rahul >>>> Today is the first day of the rest of your life ! >>>> >>>> >>>> On Sun, Jun 14, 2026 at 11:24 AM Rahul Basu <[email protected]> wrote: >>>> >>>>> Friends, >>>>> >>>>> Thanks for the interest in this, unexpected for me. I now have to sort >>>>> through the data and figure out how to organise it for public use. Give me >>>>> some time on this. >>>>> >>>>> For the moment, here's two datasets I received under RTI around >>>>> transport permits. >>>>> >>>>> *Background*: Goa runs an ore accounting software, which basically >>>>> accounts for mineral ore at different locations. To move ore from one >>>>> point >>>>> to another, a transport permit is required. This is usually for a decent >>>>> quantity, say 10,000 tons, to move from Mine X to Jetty Y. Then there are >>>>> truck trips (usually 10 tons each), and the trucks are monitored real >>>>> time. >>>>> There are also trips by barge from the jetty to the ship, and some rail as >>>>> well. When all the trips against the permit are completed, there's usually >>>>> some difference, which gets adjusted. The software provider has changed >>>>> from Megasoft to GEL. GEL software is called Bhumija. >>>>> >>>>> Mining was in three major phases: pre-2012 ban; 2015-2018; and 2024 >>>>> onwards. Ore is mined in Goa, most exported, some consumed in Goa, some >>>>> shipped by barge to JSW Dolvi, Maharashtra (recent). Ore is also imported >>>>> into Goa from overseas. Some ore mined in Karnataka and Maharashtra also >>>>> moves through Goa, mostly for export but occasionally for use in steel >>>>> plants in Goa. >>>>> >>>>> The first folder >>>>> <https://drive.google.com/drive/folders/1zeV-zquaHPq8pm6DkTjUHdIr9C_T-IBL?usp=sharing> >>>>> has >>>>> transport permits. Lots of them. From 1-Apr-2020-2025 >>>>> >>>>> The second folder >>>>> <https://drive.google.com/drive/folders/1rx21ExPmuWpWAd4HA6BgISARTVAzjsF1?usp=sharing> >>>>> is around closing stock also has some version of the manuals for the >>>>> software. I had asked for: >>>>> >>>>> a) Manuals of the Bhumija software >>>>> b) Guidelines / SOPs / instruction for monitoring ore through Bhumija >>>>> c) In excel format, for each major mineral ore transport trip after >>>>> 25-Mar-2024, permit number, carrier registration number, the source >>>>> location with time and weight at the source weighbridge and the >>>>> destination >>>>> location with time and weight >>>>> d) In excel format, closing stock of major mineral ore as of >>>>> 31-Mar-2024 and 31-Mar-2025 for all locations in Bhumija >>>>> >>>>> I have a few questions from the data: >>>>> 1) Does the ore balance at any location go negative at any point in >>>>> time? >>>>> 2) Overall statistics - major input locations, major transport routes, >>>>> total quantities, etc. >>>>> 3) Is the data complete? Is any further information needed / >>>>> desireable? >>>>> >>>>> More questions will come as other data sets get added >>>>> >>>>> Please feel free to work with this while I try to set up a better data >>>>> repository. Email me directly if you have questions on the data in the two >>>>> folders. >>>>> >>>>> Warmly >>>>> Rahul >>>>> On Saturday, June 13, 2026 at 8:59:42 PM UTC+5:30 Natasha kalra wrote: >>>>> >>>>>> Hi Rahul, >>>>>> >>>>>> Thank you for sharing this. The dataset sounds extremely valuable, >>>>>> especially given the combination of RTI-based information, government >>>>>> records, GIS layers, and transport-chain data linking leases, jetties, >>>>>> and >>>>>> shipments. >>>>>> >>>>>> I work in the broader sustainability, environmental governance, and >>>>>> data-driven policy space, and I would be interested in understanding the >>>>>> scope of the datasets and the kinds of analyses you are looking to >>>>>> undertake. I am affiliated with the Institute for Social and >>>>>> Economic Change (ISEC), Bengaluru, a premier interdisciplinary research >>>>>> institute engaged in policy-oriented research, consultancy, and >>>>>> capacity-building across areas such as sustainability, environmental >>>>>> governance, urban development, and social transformation. Through our >>>>>> work, >>>>>> we collaborate with government agencies, corporates, development >>>>>> organizations, and communities to develop evidence-based and scalable >>>>>> solutions to contemporary challenges. >>>>>> >>>>>> It would be great to connect and discuss the available data, current >>>>>> status of data cleaning and organization, and potential avenues for >>>>>> collaboration. Please let me know a convenient time for a call. >>>>>> >>>>>> Looking forward to learning more. >>>>>> >>>>>> Warm regards, >>>>>> >>>>>> Natasha >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Dr. Natasha Kalra >>>>>> >>>>>> Environmental Sustainability Consultant >>>>>> >>>>>> Initiator- BaansCraft >>>>>> >>>>>> https://www.instagram.com/baanscraft/ >>>>>> >>>>>> M: + 91 99167 70495 <+91%2099167%2070495> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Sat, Jun 13, 2026 at 6:57 PM bibhu prasad mishra < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Rahul, >>>>>>> >>>>>>> I am very interested in collaborating on this. As I am from Odisha, >>>>>>> which is a major mineral hub, I believe this is a great opportunity to >>>>>>> work >>>>>>> with this data. >>>>>>> >>>>>>> I would love to help with the data manipulation and visualization, >>>>>>> especially as you look toward the second phase involving Odisha. Please >>>>>>> let >>>>>>> me know how we can get started. >>>>>>> >>>>>>> >>>>>>> Thanks and Regards >>>>>>> *Bibhu Prasad Mishra* >>>>>>> *Civil Engineer,* >>>>>>> *B.tech, M.tech.* >>>>>>> 7894761150 <078947%2061150>, 9937512121 <099375%2012121> >>>>>>> >>>>>>> On Sat, Jun 13, 2026, 6:28 PM Arun Ganesh <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Rahul, this is a very deep topic and requires many hands to dig >>>>>>>> through (pun intended). is it possible to share links to some of the >>>>>>>> collected data so it can be explored? >>>>>>>> >>>>>>>> Just extracting data from PDF tables requires a lot of effort and >>>>>>>> it would be best to keep the data open so it can get maximum interest. >>>>>>>> Or >>>>>>>> are there reasons to not make this public given its sensitive nature? >>>>>>>> >>>>>>>> On Sat, Jun 13, 2026 at 12:58 PM Dr. C.Gajendran < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Dear Your email find interest to me; please contact me at >>>>>>>>> 9443368980 <094433%2068980>. We shall discuss further. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Dr. C. GAJENDRAN B.E (Civil) M.E >>>>>>>>> (Env.Eng.),Ph.D.,D.R.T.M.,D.I.S.,FIE(Ind.)., >>>>>>>>> IGS., ISG., >>>>>>>>> Vice Principal | Director – IQAC | Professor & Head, Civil >>>>>>>>> Engineering | Vice President IIC >>>>>>>>> *Jawaharlal College of Engineering and Technology, Palakkad* >>>>>>>>> >>>>>>>>> Mobile & Whatsup! : +91-9443368980 <+91%2094433%2068980> >>>>>>>>> skype: c_gajendran >>>>>>>>> Google Site <https://sites.google.com/view/drgajendran/home> >>>>>>>>> ORCHID : 0000-0002-2765-7075 >>>>>>>>> Vidwan <https://vidwan.inflibnet.ac.in/profile/96162> >>>>>>>>> Scopus Author ID: 57359210200 >>>>>>>>> <http://www.scopus.com/inward/authorDetails.url?authorID=57359210200&partnerID=MN8TOARS> >>>>>>>>> >>>>>>>>> Face book <https://www.facebook.com/cgajendran/> : >>>>>>>>> Twitter : dr_gajendran >>>>>>>>> Instagram : dr.gajendran >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jun 13, 2026 at 12:52 PM Rahul Basu <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi friends, >>>>>>>>>> >>>>>>>>>> I work on mining in India, primarily Goa. Over time, a lot of >>>>>>>>>> data has been collected through various public sources and RTI. >>>>>>>>>> Much of it >>>>>>>>>> is in excel, a part is in GIS form. Some of the excel is old >>>>>>>>>> government >>>>>>>>>> style, needs openrefine to cleanup. >>>>>>>>>> >>>>>>>>>> Lots of interesting analysis can be done (for example >>>>>>>>>> <https://ecoinsee.org/journal/ojs/index.php/ees/article/view/1878/355>). >>>>>>>>>> Some of data includes detailed information on transport of >>>>>>>>>> minerals from leases to jetties to ships. >>>>>>>>>> >>>>>>>>>> I'm not very good at data manipulation / data visualization and >>>>>>>>>> was looking for collaborators. >>>>>>>>>> >>>>>>>>>> If anyone is interested, please ping me. >>>>>>>>>> >>>>>>>>>> PS. Odisha seems to have good mining data as well, though much >>>>>>>>>> more data collection is needed. Can be looked at in a second phase. >>>>>>>>>> >>>>>>>>>> Warmly >>>>>>>>>> Rahul >>>>>>>>>> Today is the first day of the rest of your life ! >>>>>>>>>> Goa Foundation <http://goafoundation.org> | The Future We Need >>>>>>>>>> <http://thefutureweneed.org> | Goenchi Mati Movement >>>>>>>>>> <https://linktr.ee/goenchimati> >>>>>>>>>> https://orcid.org/0000-0002-1218-878X >>>>>>>>>> The Supreme Court and Intergenerational Equity: The Goa Mining >>>>>>>>>> Case >>>>>>>>>> <https://www.banyantreebookstore.com/product-page/the-supreme-court-of-india-and-intergenerational-equity-the-goa-mining-case> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Datameet is a community of Data Science enthusiasts in India. >>>>>>>>>> Know more about us by visiting http://datameet.org >>>>>>>>>> --- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "datameet" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion visit >>>>>>>>>> https://groups.google.com/d/msgid/datameet/CAN8Wyx_6JXrvcYQnPSw7j%3DKZg2JEWSsB66ur4SRBGaZrsJuDRQ%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/datameet/CAN8Wyx_6JXrvcYQnPSw7j%3DKZg2JEWSsB66ur4SRBGaZrsJuDRQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>>>>>> more about us by visiting http://datameet.org >>>>>>>>> --- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "datameet" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> To view this discussion visit >>>>>>>>> https://groups.google.com/d/msgid/datameet/CAMFo2vtYsS6B%3DMs_du2eE8H3%2BYqTjcUfopNhXp0%3De_37fWqqLA%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/datameet/CAMFo2vtYsS6B%3DMs_du2eE8H3%2BYqTjcUfopNhXp0%3De_37fWqqLA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>>>>> more about us by visiting http://datameet.org >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "datameet" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion visit >>>>>>>> https://groups.google.com/d/msgid/datameet/CA%2BGKQr1ph9XXnZ%3Dv5qC1ikSafjUqLu9gMCJcyqE6QX_FZ5gOHg%40mail.gmail.com >>>>>>>> <https://groups.google.com/d/msgid/datameet/CA%2BGKQr1ph9XXnZ%3Dv5qC1ikSafjUqLu9gMCJcyqE6QX_FZ5gOHg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>>>> more about us by visiting http://datameet.org >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "datameet" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>> To view this discussion visit >>>>>>> https://groups.google.com/d/msgid/datameet/CAN%3DkOZnAQb-N%3D%3DLg5kfTwCgxhG2oaorB8p3PGdcXopJnMONiFQ%40mail.gmail.com >>>>>>> <https://groups.google.com/d/msgid/datameet/CAN%3DkOZnAQb-N%3D%3DLg5kfTwCgxhG2oaorB8p3PGdcXopJnMONiFQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>> Datameet is a community of Data Science enthusiasts in India. Know >>>>> more about us by visiting http://datameet.org >>>>> --- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "datameet" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/datameet/WLHGLnl5IX0/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/datameet/c67f7584-fcbc-4a1c-83ba-dff8b0e08085n%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/datameet/c67f7584-fcbc-4a1c-83ba-dff8b0e08085n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >> Datameet is a community of Data Science enthusiasts in India. Know more >> about us by visiting http://datameet.org >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "datameet" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/datameet/WLHGLnl5IX0/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion visit >> https://groups.google.com/d/msgid/datameet/fd96996b-f85d-4bc5-8174-384c56885192n%40googlegroups.com >> <https://groups.google.com/d/msgid/datameet/fd96996b-f85d-4bc5-8174-384c56885192n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/datameet/CAN8Wyx9D2Zunt0Lxy77kg2QgPiy-qACRSk66DKet5pVrAiHojQ%40mail.gmail.com.
