On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <voze...@gridgain.com> wrote:
> I am not sure it is good idea to send requests to 3rd-party addresses from > Ignite node. Let's do not make the same mistakes again. > Agree with Vladimir. We obviously have CGI support on the website. Can someone explain why CGI is not possible to use? > > On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <anovi...@gridgain.com> > wrote: > > > We may directly send request to GA from Ignite node: > > https://developers.google.com/analytics/devguides/ > collection/protocol/v1/ > > <https://developers.google.com/analytics/devguides/ > collection/protocol/v1/ > > > > > Latest version can be received from maven central: > > https://repo1.maven.org/maven2/org/apache/ignite/ > > ignite-core/maven-metadata.xml <https://repo1.maven.org/ > > maven2/org/apache/ignite/ignite-core/maven-metadata.xml> > > > > > > > 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <dsetrak...@apache.org> > > написал(а): > > > > > > Denis, > > > > > > I am not sure I understand. We already do have CGI enabled for > > > download.cgi. Is there something else we need? > > > > > > D. > > > > > > On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dma...@gridgain.com> > wrote: > > > > > >> There is an obstacle. There is no way to execute the script using PHP > or > > >> similar sever side language and trigger GA as discussed earlier: > > >> https://issues.apache.org/jira/browse/INFRA-15182 > > >> > > >> How else can we tackle this? > > >> > > >> Denis > > >> > > >> On Thursday, September 7, 2017, Dmitriy Setrakyan < > > dsetrak...@apache.org> > > >> wrote: > > >> > > >>> I think it is safe to assume at this point that everyone is in > general > > >>> agreement, since there are no active objections. > > >>> > > >>> I have filed a ticket for the 2.3 release. Let's try to make it > happen: > > >>> https://issues.apache.org/jira/browse/IGNITE-6305 > > >>> > > >>> D. > > >>> > > >>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan < > > >> dsetrak...@apache.org > > >>> <javascript:;>> > > >>> wrote: > > >>> > > >>>> > > >>>> > > >>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani < > raul....@evosent.com > > >>> <javascript:;>> > > >>>> wrote: > > >>>> > > >>>>> Yeah, I guess that's doable as well and requires less management > > >> effort > > >>>>> than my suggestion. We could use events [1] to store payload data > > >> (e.g. > > >>>>> IP, > > >>>>> version, etc.) > > >>>> > > >>>> > > >>>> Yes, we could use events or some other similar API provided by GA. > > >>>> > > >>>> > > >>>>> What the download page CGI developed in? PHP? > > >>>>> > > >>>> > > >>>> To be honest, no clue. I guess someone in the community can figure > it > > >>> out: > > >>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html > > >>>> > > >>>> > > >>>>> However, I'm not sure whether storing this data in a 3rd party > > >> (Google) > > >>> is > > >>>>> compliant with the ASF policy. I guess it's no biggie, but if > there's > > >>>>> doubt > > >>>>> in the PMC, it's better to ask legal@. > > >>>> > > >>>> > > >>>> I am not sure there is anything to ask about. The whole Ignite > website > > >> is > > >>>> GA enabled, and all we are doing is accessing a standard web page > from > > >>> the > > >>>> Ignite web site. The information gathered from GA is available only > to > > >>> the > > >>>> Ignite PMC. Frankly, I think legal@ will be very confused by this > > >>>> question. > > >>>> > > >>>> Even ASF website itself uses GA: https://www.apache.org/ > > >>>> foundation/policies/privacy.html > > >>>> > > >>>> > > >>>>> I think Cos said it's OK; maybe Roman can pitch in. > > >>>>> > > >>>> > > >>>> Sure, would be nice to hear from Roman as well. > > >>>> > > >>>> > > >>>>> Cheers. > > >>>>> > > >>>>> [1] > > >>>>> https://developers.google.com/analytics/devguides/collection > > >>>>> /analyticsjs/events > > >>>>> > > >>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan < > > >>>>> dsetrak...@apache.org <javascript:;>> > > >>>>> wrote: > > >>>>> > > >>>>>> Raul, > > >>>>>> > > >>>>>> Could point about Javascript, it will not work because it executes > > >> in > > >>>>> the > > >>>>>> browser. This means we need a server-side script, like CGI we are > > >>> using > > >>>>> on > > >>>>>> our download page. > > >>>>>> > > >>>>>> How about this approach. We create something like > ignite-version.cgi > > >>>>> script > > >>>>>> which will invoke a call to GA and then return the latest version. > > >>>>>> > > >>>>>> This should work, right? > > >>>>>> > > >>>>>> D. > > >>>>>> > > >>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani < > > >> raul....@evosent.com > > >>> <javascript:;>> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> Hey Dmitriy and all > > >>>>>>> > > >>>>>>> Also, since we have GA enabled for the website, we can track how > > >>> many > > >>>>>> times > > >>>>>>>> this page was accessed, which will be equal to the number of > > >>> starts. > > >>>>>> This > > >>>>>>>> way, the counter information is tracked and monitored by the > > >>> Ignite > > >>>>>> PMC. > > >>>>>>> > > >>>>>>> > > >>>>>>> Unfortunately this won't work because GA is loaded via Javascript > > >> on > > >>>>> the > > >>>>>>> browser, so Google will never receive the page hit. > > >>>>>>> > > >>>>>>> Given the constraints, the most viable solution is an HTTPS > > >> endpoint > > >>>>>>> running on ASF infra that Ignite invokes via a GET or POST > > >> request. > > >>>>> The > > >>>>>>> simplest thing is to write each request in a log file, along with > > >>> the > > >>>>>>> timestamp, the version reported by the client, maybe the IP (not > > >>> sure > > >>>>>> about > > >>>>>>> the ASF rules about this concerning privacy, I guess it's OK if > > >> you > > >>>>> make > > >>>>>> it > > >>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node > > >>> creates > > >>>>> on > > >>>>>>> first startup or something. > > >>>>>>> > > >>>>>>> This endpoint would need some basic DDoS protection and > > >> blacklisting > > >>>>> to > > >>>>>>> prevent data spoofing. > > >>>>>>> > > >>>>>>> If we'll be implementing this endpoint anyway, then there's no > > >> point > > >>>>>>> placing another file on Git or elsewhere for reporting the latest > > >>>>>> versions: > > >>>>>>> the endpoint itself can return them. > > >>>>>>> > > >>>>>>> WDYT? > > >>>>>>> > > >>>>>>> Cheers. > > >>>>>>> > > >>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan < > > >>>>>> dsetrak...@apache.org <javascript:;>> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Cos, Raul, > > >>>>>>>> > > >>>>>>>> Thanks for the feedback. I completely agree about Maven Central > > >>>>> being a > > >>>>>>> 3rd > > >>>>>>>> party repo (did not think about that initially). All your > > >>>>> suggestions > > >>>>>>> make > > >>>>>>>> sense, but I would like to keep it as simple as possible, and so > > >>> far > > >>>>>>>> everything suggested required GIT dependencies and extra work. > > >>>>>>>> > > >>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite > > >>>>>> website > > >>>>>>>> which will have only the latest version. Every time a node > > >> starts, > > >>>>> it > > >>>>>>>> receives the latest version from the page and suggests that > > >> users > > >>>>>> upgrade > > >>>>>>>> if needed. > > >>>>>>>> > > >>>>>>>> Also, since we have GA enabled for the website, we can track how > > >>>>> many > > >>>>>>> times > > >>>>>>>> this page was accessed, which will be equal to the number of > > >>> starts. > > >>>>>> This > > >>>>>>>> way, the counter information is tracked and monitored by the > > >>> Ignite > > >>>>>> PMC. > > >>>>>>>> > > >>>>>>>> This approach looks pretty innocent to me and everything is kept > > >>> and > > >>>>>>>> managed within Apache. > > >>>>>>>> > > >>>>>>>> Thoughts? > > >>>>>>>> > > >>>>>>>> D. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik < > > >>>>> c...@apache.org <javascript:;>> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> I agree with Raul. > > >>>>>>>>> - providing a ping-back address to a 3rd party might be frown > > >>>>> upon by > > >>>>>>>> some. > > >>>>>>>>> And might have a consequences like stats collection about > > >>> users' > > >>>>>>>>> infrastructure. > > >>>>>>>>> - checking an ASF git-repo is easy and won't download any > > >> binary > > >>>>>> data: > > >>>>>>>>> everything is clear text and could be easily monitored by > > >> any > > >>> of > > >>>>>> the > > >>>>>>>>> network > > >>>>>>>>> diagnostic tools, shall it be required. But it involves a > > >> bit > > >>> of > > >>>>>> the > > >>>>>>>>> release > > >>>>>>>>> discipline. > > >>>>>>>>> - the binary data download in the runtime is my main concern. > > >>>>> This is > > >>>>>>> the > > >>>>>>>>> vector of MMA. What if someone gains the control over the > > >>>>>> repository > > >>>>>>>> and > > >>>>>>>>> replaces the file with some malicious content. > > >>>>>>>>> > > >>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a > > >> call > > >>>>> to > > >>>>>> an > > >>>>>>>>> external script to check upon the atest software version > > >>> available > > >>>>>> for > > >>>>>>>>> download. In the past, the endpoint was running on a 3rd party > > >>>>>> server, > > >>>>>>> I > > >>>>>>>>> believe the best way would be to put this script on ASF infra > > >>> and > > >>>>>> have > > >>>>>>>> the > > >>>>>>>>> "update checker" running in a completely controlled > > >> environment. > > >>>>>>>> Actually, > > >>>>>>>>> I > > >>>>>>>>> recall we had this very discussion during the Incubation; I > > >> can > > >>>>>>> probably > > >>>>>>>>> dig > > >>>>>>>>> out the corresponding thread. > > >>>>>>>>> > > >>>>>>>>> Thoughts? > > >>>>>>>>> Cok > > >>>>>>>>> > > >>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote: > > >>>>>>>>>> Hey guys > > >>>>>>>>>> > > >>>>>>>>>> In my opinion, maven.org is still owned by a third party > > >>>>>> (Sonatype). > > >>>>>>>> We > > >>>>>>>>>> don't know what kind of data analysis or intelligence > > >>> extraction > > >>>>>> they > > >>>>>>>>> run. > > >>>>>>>>>> > > >>>>>>>>>> If Ignite servers all over the world were hitting maven.org > > >>>>>>>> periodically > > >>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a > > >> clear > > >>>>>>>> indication > > >>>>>>>>>> of who is running an Ignite server. > > >>>>>>>>>> > > >>>>>>>>>> They could reverse lookup the IP address and find out what > > >>>>>>> corporation > > >>>>>>>> it > > >>>>>>>>>> is. > > >>>>>>>>>> > > >>>>>>>>>> How about having Ignite check the ASF Git directly? > > >>>>>>>>>> > > >>>>>>>>>> We could use the Git ssh API, but that would require a new > > >>>>>>> dependency, > > >>>>>>>>>> which I advise against. > > >>>>>>>>>> > > >>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git > > >>>>> tags: > > >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git > > >>>>>>>>>> > > >>>>>>>>>> Another option is to place a txt file in the root of the > > >>> master > > >>>>>>> branch > > >>>>>>>>> (e.g > > >>>>>>>>>> LATEST), containing a list of the latest GA versions for > > >> each > > >>>>> major > > >>>>>>>>> version > > >>>>>>>>>> line (1.x, 2.x). > > >>>>>>>>>> > > >>>>>>>>>> I would advocate this last option, but it requires somebody > > >>>>>>> remembering > > >>>>>>>>> to > > >>>>>>>>>> update the file with every release, unless we automate it > > >>> with a > > >>>>>>> Maven > > >>>>>>>>>> plugin. > > >>>>>>>>>> > > >>>>>>>>>> Hope that helps! > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dma...@apache.org > > >>> <javascript:;>> wrote: > > >>>>>>>>>> > > >>>>>>>>>> I see nothing wrong with this approach. > > >>>>>>>>>> > > >>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is > > >> it > > >>>>> good > > >>>>>>> to > > >>>>>>>>> go? > > >>>>>>>>>> > > >>>>>>>>>> — > > >>>>>>>>>> Denis > > >>>>>>>>>> > > >>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan < > > >>>>>>>> dsetrak...@apache.org <javascript:;> > > >>>>>>>>>> > > >>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket > > >> on > > >>>>> it? > > >>>>>>>>>>> > > >>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan < > > >>>>>>>>> dsetrak...@apache.org <javascript:;>> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Igniters, > > >>>>>>>>>>>> > > >>>>>>>>>>>> There has been lots of talk of proposals about various > > >>> usage > > >>>>>>> metrics > > >>>>>>>>> for > > >>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect > > >>> the > > >>>>>> topic > > >>>>>>>> and > > >>>>>>>>>>>> propose something very simple and non-intrusive. > > >>>>>>>>>>>> > > >>>>>>>>>>>> 1. Update Checker > > >>>>>>>>>>>> The main purpose of the update checker is not to collect > > >>>>>> metrics, > > >>>>>>>> but > > >>>>>>>>> to > > >>>>>>>>>>>> notify users about a new version of Ignite by accessing > > >>>>>> maven.org > > >>>>>>>> and > > >>>>>>>>>>>> getting the version out of the metadata file: > > >>>>>>>>>>>> http://repo2.maven.org/maven2/ > > >>> org/apache/ignite/ignite-core/ > > >>>>>>>>>>>> maven-metadata.xml > > >>>>>>>>>>>> > > >>>>>>>>>>>> This way we do not send any information anywhere and, at > > >>> the > > >>>>>> same > > >>>>>>>>> time, > > >>>>>>>>>>>> urge our users to download and start using the latest > > >>>>> version of > > >>>>>>>>> Ignite. > > >>>>>>>>>>>> > > >>>>>>>>>>>> 2. Startup Counter > > >>>>>>>>>>>> This piece is optional, but we can also get an insight in > > >>> how > > >>>>>> many > > >>>>>>>>> times > > >>>>>>>>>> a > > >>>>>>>>>>>> certain Ignite release gets started. This is just a cool > > >>>>> metric > > >>>>>>> for > > >>>>>>>>> the > > >>>>>>>>>>>> community to gauge the project popularity. You can think > > >> of > > >>>>> it > > >>>>>> as > > >>>>>>>> of a > > >>>>>>>>>> page > > >>>>>>>>>>>> visit counter shown on many websites. We can even decide > > >> to > > >>>>>>> display > > >>>>>>>>> this > > >>>>>>>>>>>> counter on the Ignite website as well. > > >>>>>>>>>>>> > > >>>>>>>>>>>> To do this, we can simply add a JAR in maven for every > > >>>>> release, > > >>>>>>> e.g. > > >>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte. > > >>>>> Every > > >>>>>>> time > > >>>>>>>>> an > > >>>>>>>>>>>> Ignite node starts, it will download this JAR in the > > >>>>> background. > > >>>>>>>> Then > > >>>>>>>>> we > > >>>>>>>>>>>> will be able to view the number of the total downloads > > >> for > > >>>>> this > > >>>>>>> JAR > > >>>>>>>> in > > >>>>>>>>>>>> Maven Central, which is essentially the number of starts > > >> of > > >>>>>> Ignite > > >>>>>>>>> nodes. > > >>>>>>>>>>>> > > >>>>>>>>>>>> *Note that neither of the above suggestions require > > >> Ignite > > >>> to > > >>>>>> send > > >>>>>>>> or > > >>>>>>>>>>>> track any user information whatsoever.* > > >>>>>>>>>>>> > > >>>>>>>>>>>> Please reply suggesting weather you are OK with this > > >>>>> approach. > > >>>>>>>>>>>> > > >>>>>>>>>>>> D. > > >>>>>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>> > > >> > > > > >
