Re: [datameet] Local sub urban trains

[Samuel Rajkumar]

I used to take trains from Lottegollahalli to Banaswadi regularly last
year. I know people who travel regularly between Lottegollahalli and
Karmelram / Belandur Road. I use
http://indiarailinfo.com/trains/passengera lot for schedules.


On Sun, Mar 2, 2014 at 7:33 PM, Thejesh GN  wrote:

> - do you use it? Like from kr puram to yeshwanthpur ?
>
> - do you know anybody who uses it?
>
> - do they use passengers or express? What are the rules
>
> - can we build a system which makes it easy to use.
>
> - any other ideas
>
> - next scraping would be metro n local train data.
>
> --
> Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
> http://thejeshgn.com
> GPG ID :  0xBFFC8DD3C06DD6B0
>
> --
> For more details about this list
> http://datameet.org/discussions/
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[datameet] Local sub urban trains

[Thejesh GN]

- do you use it? Like from kr puram to yeshwanthpur ?

- do you know anybody who uses it?

- do they use passengers or express? What are the rules

- can we build a system which makes it easy to use.

- any other ideas

- next scraping would be metro n local train data.

--
Thejesh GN ⏚ ತೇಜೇಶ್ ಜಿ.ಎನ್
http://thejeshgn.com
GPG ID :  0xBFFC8DD3C06DD6B0

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Sanjay Bhangar]

Hey,

Great to see this conversation happening, hope to be able to
contribute. My two bits inline -

On Sun, Mar 2, 2014 at 3:15 PM, Gora Mohanty  wrote:
> On 2 March 2014 14:51, Sajjad Anwar  wrote:
> [...]
>> Hey, you don't necessarily need geodjango for these. You can write simple
>> spatial queries in PostGIS.
>> And they aren't very expensive. For example -
>> http://postgis.refractions.net/documentation/manual-2.0/ST_Within.html
>
> GeoDjango queries translate to postgis if one is using a postgresql
> back-end, so I do not think that there is much overhead there. GeoDjango
> also does a lot more, including providing an abstraction layer for GIS, so
> that I would prefer using it to coding directly in postgis. The alternatives
> that I was thinking of were along the lines of spatial search with 
> Solr/Lucene.
> These should be faster in principle, but we have not yet benchmarked
> them against postgis/GeoDjango.
>

I wouldn't worry about performance before it's a concern - I think
PostGIS is fairly performant. If one does land up hosting a lot of
data, I'd highly recommend ElasticSearch for its geo support and
scalability, but there's probably a few options. I'd also highly
recommend django-haystack [0] - in most cases, this should let one
fairly easily build out your django project, and add a performant
search indexer when one needs it. I'll be glad to help. Any-how, my
two bits would be "don't premature optimize" - I think building
something out using GeoDjango should be fairly clean and
straightforward, and I think django-haystack provides a clean
migration path forward if one does need to scale / use a specialized
indexing tool like Lucene.

Also, for Django APIs, I've been looking at django-rest-framework [1]
recently, and am quite liking it. It seemed a bit of overhead as
compared to just writing vanilla view functions for end-points, but
gives a really nice, auto-documented API frame-work with a nice
in-browser testing page, multiple formats, etc. I had to do some
tweaking to get it to output GeoJSON - any-how, if you are going down
this path and want a hand / second set of eyes, let me know.

> I would again argue that the first thing to do would be to have a stable 
> system
> that provides an API layer over raw data pulled from various sources. GIS,
> and such queries would be meta-queries over such raw data, and should be
> done later, IMHO.
>

About other sort of aggregation / meta-queries, I agree, but I think
basic GIS data can be quite fundamental to this sort of data. Its nice
to be able to query location data, and for data-sets that have
location data, I think would be nice to support? Thankfully, adding
basic geo support using GeoDjango is fairly straightfoward - I agree
that performance maybe an issue, but I think if the site is getting
that many users / has that much data, one could work out more complex
solutions then?

Anyways, let the bike-shedding not get to you, Thej :) - do whatever's
easiest / makes sense - do let me know if I can help / contribute /
when there's code up, etc.

All the best!
-Sanjay

[0] http://haystacksearch.org/
[1] http://www.django-rest-framework.org/

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Gora Mohanty]

On 2 March 2014 14:51, Sajjad Anwar  wrote:
[...]
> Hey, you don't necessarily need geodjango for these. You can write simple
> spatial queries in PostGIS.
> And they aren't very expensive. For example -
> http://postgis.refractions.net/documentation/manual-2.0/ST_Within.html

GeoDjango queries translate to postgis if one is using a postgresql
back-end, so I do not think that there is much overhead there. GeoDjango
also does a lot more, including providing an abstraction layer for GIS, so
that I would prefer using it to coding directly in postgis. The alternatives
that I was thinking of were along the lines of spatial search with Solr/Lucene.
These should be faster in principle, but we have not yet benchmarked
them against postgis/GeoDjango.

I would again argue that the first thing to do would be to have a stable system
that provides an API layer over raw data pulled from various sources. GIS,
and such queries would be meta-queries over such raw data, and should be
done later, IMHO.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Sajjad Anwar]

On Sun, Mar 2, 2014 at 2:27 PM, Thejesh GN  wrote:

>
>
> On Sun, Mar 2, 2014 at 2:21 PM, Gora Mohanty  wrote:
>
>> On 2 March 2014 13:13, Thejesh GN  wrote:
>> >
>> > On Mar 1, 2014 7:59 PM, "Gautam John"  wrote:
>> >>
>> >> On Sat, Mar 1, 2014 at 7:34 PM, Gora Mohanty 
>> wrote:
>> >>
>> >> > idea of having a way to get data from the Excel files, and other
>> >> > sources, on
>> >> > data.gov.in and other places, and providing real API-based access,
>> and
>> >> > have
>> >> > some thoughts on this. Should we discuss this here, or take it
>> offline.
>> >>
>> >> Wouldn't something like CKAN work?
>> >>
>> >> http://docs.ckan.org/en/latest/api/index.html
>> >
>> > I had not checked the CKAN in a long time. It looks very interesting. It
>> > doesn't say anything about GIS part. I am very interested in GIS. I will
>> > probably install over the weekend and let group explore.
>>
>> We have extensively used GIS with GeoDjango. While GIS is doable,
>> maybe we should agree upon what the primary purpose of the site should
>> be. What I thought that you were proposing was a site that acts as a
>> central location where data from multiple Indian data sources are
>> accumulated,
>> and provided to users via an API.
>>
>> While we are also interested in GIS, visualisation, and reporting, to my
>> mind
>> that should be a secondary focus for this site.
>>
>>
> My focus is to deliver bangalore data through API. that includes location
> based query too
> - For example given lat, long what are the nearest bus stops.
> - Given a lat, long what are the nearest schools
> - Given a lat, long whats the AC, PB and WARD
> etc
>
> There are the things that can be consumed by an app directly.
>

Hey, you don't necessarily need geodjango for these. You can write simple
spatial queries in PostGIS.
And they aren't very expensive. For example -
http://postgis.refractions.net/documentation/manual-2.0/ST_Within.html

I'll be more than happy to help if you need support on the spatial data
front.

Cheers,
Sajjad.

>
>
>
>
>
>
>> Regards,
>> Gora
>>
>> --
>> For more details about this list
>> http://datameet.org/discussions/
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "datameet" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to datameet+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> For more details about this list
> http://datameet.org/discussions/
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
Sajjad Anwar http://geohacker.in 

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Gora Mohanty]

On 2 March 2014 14:27, Thejesh GN  wrote:
[...]
> My focus is to deliver bangalore data through API. that includes location
> based query too
> - For example given lat, long what are the nearest bus stops.
> - Given a lat, long what are the nearest schools
> - Given a lat, long whats the AC, PB and WARD
> etc
>
> There are the things that can be consumed by an app directly.

OK, that makes sense. GIS queries are expensive though, and we
will have to worry about how best to handle these.

If you can check out CKAN, I will start articulating our ideas on
how to crowd-source data-import from poorly-structured CSV/Excel
files which unfortunately seems to be the preferred format for many
data sites.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Thejesh GN]

On Sun, Mar 2, 2014 at 2:21 PM, Gora Mohanty  wrote:

> On 2 March 2014 13:13, Thejesh GN  wrote:
> >
> > On Mar 1, 2014 7:59 PM, "Gautam John"  wrote:
> >>
> >> On Sat, Mar 1, 2014 at 7:34 PM, Gora Mohanty 
> wrote:
> >>
> >> > idea of having a way to get data from the Excel files, and other
> >> > sources, on
> >> > data.gov.in and other places, and providing real API-based access,
> and
> >> > have
> >> > some thoughts on this. Should we discuss this here, or take it
> offline.
> >>
> >> Wouldn't something like CKAN work?
> >>
> >> http://docs.ckan.org/en/latest/api/index.html
> >
> > I had not checked the CKAN in a long time. It looks very interesting. It
> > doesn't say anything about GIS part. I am very interested in GIS. I will
> > probably install over the weekend and let group explore.
>
> We have extensively used GIS with GeoDjango. While GIS is doable,
> maybe we should agree upon what the primary purpose of the site should
> be. What I thought that you were proposing was a site that acts as a
> central location where data from multiple Indian data sources are
> accumulated,
> and provided to users via an API.
>
> While we are also interested in GIS, visualisation, and reporting, to my
> mind
> that should be a secondary focus for this site.
>
>
My focus is to deliver bangalore data through API. that includes location
based query too
- For example given lat, long what are the nearest bus stops.
- Given a lat, long what are the nearest schools
- Given a lat, long whats the AC, PB and WARD
etc

There are the things that can be consumed by an app directly.






> Regards,
> Gora
>
> --
> For more details about this list
> http://datameet.org/discussions/
> ---
> You received this message because you are subscribed to the Google Groups
> "datameet" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to datameet+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [datameet] API for Bangalore

[Gora Mohanty]

On 2 March 2014 13:13, Thejesh GN  wrote:
>
> On Mar 1, 2014 7:59 PM, "Gautam John"  wrote:
>>
>> On Sat, Mar 1, 2014 at 7:34 PM, Gora Mohanty  wrote:
>>
>> > idea of having a way to get data from the Excel files, and other
>> > sources, on
>> > data.gov.in and other places, and providing real API-based access, and
>> > have
>> > some thoughts on this. Should we discuss this here, or take it offline.
>>
>> Wouldn't something like CKAN work?
>>
>> http://docs.ckan.org/en/latest/api/index.html
>
> I had not checked the CKAN in a long time. It looks very interesting. It
> doesn't say anything about GIS part. I am very interested in GIS. I will
> probably install over the weekend and let group explore.

We have extensively used GIS with GeoDjango. While GIS is doable,
maybe we should agree upon what the primary purpose of the site should
be. What I thought that you were proposing was a site that acts as a
central location where data from multiple Indian data sources are accumulated,
and provided to users via an API.

While we are also interested in GIS, visualisation, and reporting, to my mind
that should be a secondary focus for this site.

Regards,
Gora

-- 
For more details about this list
http://datameet.org/discussions/
--- 
You received this message because you are subscribed to the Google Groups 
"datameet" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to datameet+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.