Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.8.3
 Component:  Default  |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by sbl):

 Is there anything left here? Can the ticket be closed? Installation
 through proxies should work now...

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:
 Component:  Default  |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by mmetz):

 Replying to [comment:17 pmav99]:
 > {{{requests}}} need to be removed from {{{REQUIREMENTS.html}}} and the
 {{{Dockerfile}}}, too.

 Done in trunk r74454

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Default  |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+
Changes (by martinl):

 * component:  Addons => Default


-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by pmav99):

 {{{requests}}} need to be removed from {{{REQUIREMENTS.html}}} and the
 {{{Dockerfile}}}, too.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[Markus Neteler]

(just a comment on testing)

On Sun, Apr 14, 2019 at 5:49 AM GRASS GIS  wrote:
...
>  As a side note, since we are moving to GitHub and we expect to get more
>  pull requests, we should be more strict about what we accept. People
>  typically care about getting a specific issue fixed, but this will often
>  introduce more mess in the code or even break other functionality.

Yes, we need to have tests for much more code and instant testing.

Markus


> --
> Ticket URL: 
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by annakrat):

 If somebody could test it with proxy, that would be great, I couldn't do
 that now. This should work with Python 2/3, it uses six package (which we
 need to require unlike requests).

 If this works, we need to revert r74121.

 As a side note, since we are moving to GitHub and we expect to get more
 pull requests, we should be more strict about what we accept. People
 typically care about getting a specific issue fixed, but this will often
 introduce more mess in the code or even break other functionality.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by annakrat):

 In [changeset:"74374" 74374]:
 {{{
 #!CommitTicketReference repository="" revision="74374"
 g.extension: avoid python requests import, see #3179
 }}}

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by mmetz):

 Replying to [comment:13 mlennert]:
 > Replying to [comment:12 neteler]:
 > > Replying to [comment:11 mmetz]:
 > > > Replying to [comment:10 neteler]:
 > > > > Fix attempt via https://github.com/GRASS-GIS/grass-ci/pull/6
 submitted as r74116.
 > > > > Note the different proxy syntax in the manual page.
 > > > >
 > > > > If ok, to be backported.
 > > >
 > > > Note, this is introducing a new python dependency, on RHEL 7 and
 derivates, this is python-requests.
 > >
 > > Yes, I realized that today.
 > >
 > > > Do we have a list of required python 2 components and required
 python 3 components that can be updated?
 > >
 > > Also yes. See r74121, it is REQUIREMENTS.html.
 > >
 > > > Ideally such a list would also indicate which python components are
 required only for the GUI (systems such as HPC may use GRASS without GUI).
 > >
 > > That's more or less indicated in that file.
 > >
 > > So, I am undecided now about the backport.
 >
 > I would plead for reworking this to use urllib2 (Python 2) which is
 already imported in the module and which seems to work according to madi.

 +1
 >
 > Let's try to avoid the endless dependency hell just out of easy
 availability of small helper packages. See the [https://hackernoon.com
 /whats-really-wrong-with-node-modules-and-why-this-is-your-fault-
 8ac9fa893823 experience] with npm in the Javascript world as a case in
 point. I would plead to keep most modules down to standard python library
 dependency, unless there really is no other way.

 +1

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by mlennert):

 Replying to [comment:12 neteler]:
 > Replying to [comment:11 mmetz]:
 > > Replying to [comment:10 neteler]:
 > > > Fix attempt via https://github.com/GRASS-GIS/grass-ci/pull/6
 submitted as r74116.
 > > > Note the different proxy syntax in the manual page.
 > > >
 > > > If ok, to be backported.
 > >
 > > Note, this is introducing a new python dependency, on RHEL 7 and
 derivates, this is python-requests.
 >
 > Yes, I realized that today.
 >
 > > Do we have a list of required python 2 components and required python
 3 components that can be updated?
 >
 > Also yes. See r74121, it is REQUIREMENTS.html.
 >
 > > Ideally such a list would also indicate which python components are
 required only for the GUI (systems such as HPC may use GRASS without GUI).
 >
 > That's more or less indicated in that file.
 >
 > So, I am undecided now about the backport.

 I would plead for reworking this to use urllib2 (Python 2) which is
 already imported in the module and which seems to work according to madi.
 IIUC, authenticated proxy handling is part of urllib in Python 3, or ?

 Let's try to avoid the endless dependency hell just out of easy
 availability of small helper packages. See the [https://hackernoon.com
 /whats-really-wrong-with-node-modules-and-why-this-is-your-fault-
 8ac9fa893823 experience] with npm in the Javascript world as a case in
 point. I would plead to keep most modules down to standard python library
 dependency, unless there really is no other way.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by neteler):

 Replying to [comment:11 mmetz]:
 > Replying to [comment:10 neteler]:
 > > Fix attempt via https://github.com/GRASS-GIS/grass-ci/pull/6 submitted
 as r74116.
 > > Note the different proxy syntax in the manual page.
 > >
 > > If ok, to be backported.
 >
 > Note, this is introducing a new python dependency, on RHEL 7 and
 derivates, this is python-requests.

 Yes, I realized that today.

 > Do we have a list of required python 2 components and required python 3
 components that can be updated?

 Also yes. See r74121, it is REQUIREMENTS.html.

 > Ideally such a list would also indicate which python components are
 required only for the GUI (systems such as HPC may use GRASS without GUI).

 That's more or less indicated in that file.

 So, I am undecided now about the backport.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by mmetz):

 Replying to [comment:10 neteler]:
 > Fix attempt via https://github.com/GRASS-GIS/grass-ci/pull/6 submitted
 as r74116.
 > Note the different proxy syntax in the manual page.
 >
 > If ok, to be backported.

 Note, this is introducing a new python dependency, on RHEL 7 and
 derivates, this is python-requests.

 Do we have a list of required python 2 components and required python 3
 components that can be updated? Ideally such a list would also indicate
 which python components are required only for the GUI (systems such as HPC
 may use GRASS without GUI).

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.5
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by neteler):

 Fix attempt via https://github.com/GRASS-GIS/grass-ci/pull/6 submitted as
 r74116.
 Note the different proxy syntax in the manual page.

 If ok, to be backported.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by mmetz):

 Replying to [ticket:3179 madi]:
 > Related to https://lists.osgeo.org/pipermail/grass-
 user/2016-October/075337.html
 >
 > when i try to use g.extension behind a proxy, following the example from
 the manual, it gives me the following error:
 >
 > {{{
 > g.extension ext=i.segment.hierarchical proxy="http://xx.xxx.xxx.xx:;
 > Traceback (most recent call last):
 >   File "/usr/local/grass-7.3.svn/scripts/g.extension", line 1730, in
 
 > sys.exit(main())
 >   File "/usr/local/grass-7.3.svn/scripts/g.extension", line 1675, in
 main
 > for ptype, purl in (p.split('=') for p in
 options['proxy'].split(',')):
 > ValueError: need more than 1 value to unpack
 > }}}
 >
 > Using the form suggested by Pietro:
 >
 > {{{
 > proxy="http=,ftp="
 > }}}
 >
 > {{{
 > g.extension ext=i.segment.hierarchical
 proxy="http=http://xx.xxx.xxx.xx:;
 > }}}
 >
 > gives:
 >
 > {{{
 > Fetching  from GRASS GIS Addons repository (be
 > patient)...
 > ERROR: Extension  not found
 > }}}
 >
 > which is exactly the same as if I don't indicate the proxy at all.

 According to Python docs, "Proxies which require authentication for use
 are not currently supported; this is considered an implementation
 limitation" in urllib, and `g.extension` uses urllib, not urllib2 to open
 urls.

 >
 > FWIW, I normally use urllib2 to handle the proxy. However I have some
 difficulties to apply this simple method to the 1700+ rows code (handling
 several different addons sources for several os's) of g.extension, but I'm
 behind a proxy and I'm available to test if you have a patch.
 >
 > Here's what I do in python:
 >
 > {{{
 > import urllib2
 > import os
 > proxy = urllib2.ProxyHandler({'http':os.environ['HTTP_PROXY']})
 > opener = urllib2.build_opener(proxy)
 > urllib2.install_opener(opener)
 > urllib2.urlopen('http://www.google.com')
 > }}}

 That should also work for `g.extension`, including authentification.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by madi):

 Can't check this now, as I'm currently not behind a proxy. Operating
 system at the time of reporting was CentOS.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by neteler):

 Please indicate the operating system, the output of `g.version -reg` and
 the Python version.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by neteler):

 Proxy ticket also in #1434

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
+-
 Reporter:  madi|  Owner:  grass-dev@…
 Type:  defect  | Status:  new
 Priority:  normal  |  Milestone:  7.4.0
Component:  Addons  |Version:  unspecified
 Keywords:  g.extension, proxy, addons  |CPU:  Unspecified
 Platform:  Unspecified |
+-
 Related to https://lists.osgeo.org/pipermail/grass-
 user/2016-October/075337.html

 when i try to use g.extension behind a proxy, following the example from
 the manual, it gives me the following error:

 {{{
 g.extension ext=i.segment.hierarchical proxy="http://xx.xxx.xxx.xx:;
 Traceback (most recent call last):
   File "/usr/local/grass-7.3.svn/scripts/g.extension", line 1730, in
 
 sys.exit(main())
   File "/usr/local/grass-7.3.svn/scripts/g.extension", line 1675, in main
 for ptype, purl in (p.split('=') for p in
 options['proxy'].split(',')):
 ValueError: need more than 1 value to unpack
 }}}

 Using the form suggested by Pietro:

 {{{
 proxy="http=,ftp="
 }}}

 {{{
 g.extension ext=i.segment.hierarchical
 proxy="http=http://xx.xxx.xxx.xx:;
 }}}

 gives:

 {{{
 Fetching  from GRASS GIS Addons repository (be
 patient)...
 ERROR: Extension  not found
 }}}

 which is exactly the same as if I don't indicate the proxy at all.

 FWIW, I normally use urllib2 to handle the proxy. However I have some
 difficulties to apply this simple method to the 1700+ rows code (handling
 several different addons sources for several os's) of g.extension, but I'm
 behind a proxy and I'm available to test if you have a patch.

 Here's what I do in python:

 {{{
 import urllib2
 import os
 proxy = urllib2.ProxyHandler({'http':os.environ['HTTP_PROXY']})
 opener = urllib2.build_opener(proxy)
 urllib2.install_opener(opener)
 urllib2.urlopen('http://www.google.com')
 }}}

 Thanks

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev