[pgadmin-hackers] pgAdmin 4 commit: Fix column addition regression test.

[Dave Page]

Fix column addition regression test.

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=3ea8a7b211351b57646f7aec108f57623a40bb26
Author: Murtuza Zabuawala 

Modified Files
--
.../servers/databases/schemas/tables/column/tests/test_column_add.py| 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] [pgAdmin4][PATCH] To fix the issue in handling of timestamp type

[Dave Page]

Thanks, applied.

On Mon, Feb 6, 2017 at 6:12 AM, Murtuza Zabuawala
 wrote:
> Hi Dave,
>
> Please find a fix for the same, attribute length was set to False instead of
> None.
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> On Mon, Feb 6, 2017 at 11:18 AM, Murtuza Zabuawala
>  wrote:
>>
>> Sure, Checking.
>>
>> --
>> Regards,
>> Murtuza Zabuawala
>> EnterpriseDB: http://www.enterprisedb.com
>> The Enterprise PostgreSQL Company
>>
>> On Sat, Feb 4, 2017 at 7:34 PM, Dave Page  wrote:
>>>
>>> Hi Murtuza,
>>>
>>> I clearly neglected to run the regression tests with this patch, and
>>> unfortunately it looks like it broke them. Can you look at this ASAP
>>> please?
>>>
>>> runTest
>>> (pgadmin.browser.server_groups.servers.databases.schemas.tables.column.tests.test_column_add.ColumnAddTestCase)
>>> This function will add column under table node. (Add table Node URL)
>>> ... 2017-02-04 14:43:55,191: ERROR pgadmin: Failed to execute query
>>> (execute_scalar) for the server #4 - DB:test_db_9e176 (Query-id:
>>> 4792242):
>>> Error Message:ERROR:  syntax error at or near "False"
>>> LINE 2: ADD COLUMN test_column_add_f0d5d char(False);
>>>   ^
>>> FAIL
>>>
>>> As far as I can see, the app works fine - it's only the test that
>>> broke (probably because the application uses type name aliases (e.g.
>>> character) rather than the base type name that the test is using
>>> ("char").
>>>
>>> Thanks.
>>>
>>> On Fri, Feb 3, 2017 at 1:52 PM, Dave Page  wrote:
>>> > Thanks - patch applied.
>>> >
>>> > On Fri, Feb 3, 2017 at 11:46 AM, Murtuza Zabuawala
>>> >  wrote:
>>> >> Hi,
>>> >>
>>> >> Please find updates patch for the same.
>>> >> RM#2076
>>> >>
>>> >> --
>>> >> Regards,
>>> >> Murtuza Zabuawala
>>> >> EnterpriseDB: http://www.enterprisedb.com
>>> >> The Enterprise PostgreSQL Company
>>> >>
>>> >> On Wed, Feb 1, 2017 at 3:08 PM, Dave Page  wrote:
>>> >>>
>>> >>> Hi
>>> >>>
>>> >>> On Tue, Jan 31, 2017 at 5:19 AM, Murtuza Zabuawala
>>> >>>  wrote:
>>> >>> > Hi Dave,
>>> >>> >
>>> >>> > PFA updated patch.
>>> >>>
>>> >>> This seems to display "timestamp(0) with[out] timezone" columns
>>> >>> correctly in both the properties panel and dialog now, but the size
>>> >>> is
>>> >>> still ignored if I try to add a new column through the table or
>>> >>> column
>>> >>> dialogue.
>>> >>>
>>> >>> --
>>> >>> Dave Page
>>> >>> Blog: http://pgsnake.blogspot.com
>>> >>> Twitter: @pgsnake
>>> >>>
>>> >>> EnterpriseDB UK: http://www.enterprisedb.com
>>> >>> The Enterprise PostgreSQL Company
>>> >>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Dave Page
>>> > Blog: http://pgsnake.blogspot.com
>>> > Twitter: @pgsnake
>>> >
>>> > EnterpriseDB UK: http://www.enterprisedb.com
>>> > The Enterprise PostgreSQL Company
>>>
>>>
>>>
>>> --
>>> Dave Page
>>> Blog: http://pgsnake.blogspot.com
>>> Twitter: @pgsnake
>>>
>>> EnterpriseDB UK: http://www.enterprisedb.com
>>> The Enterprise PostgreSQL Company
>>
>>
>



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] [pgAdmin4][PATCH] To fix graphical explain for Insert/Update/Delete quries

[Dave Page]

Hi

On Mon, Feb 6, 2017 at 5:40 AM, Murtuza Zabuawala
 wrote:
> Hi,
>
> PFA patch to fix the issue with Graphical Explain plan functionality which
> was breaking for INSERT/UPDATE/DELETE explain plans.
> RM#2133
>
> Issue: It was due to typo in imageMapper object.
>
> Please review.

It's a definite improvement, but still isn't quite right - see the
attached screenshot for example; the node name should be pem.agent
shouldn't it (or 'Delete' or 'Delete from pem.agent')?


-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] [pgAdmin4][PATCH] To handle "char" type in column node

[Murtuza Zabuawala]

Hi,

PFA minor patch to handle "char" type while creating columns in Column node.

*Issue:* simplejson parse '*"char"*' as valid json, hence when we perform
simplejson.loads(request.args) operation '"char"' gets converted to 'char'
I had to handle it manually as of now, I did not find any other workaround
to handle this issue, I went over several SO question most of them
suggested manual handling.
This behavior is only in Column node, table & type nodes are not affectted
by .loads() method as we send data as column collection.
RM#2152

*Snippet of issue:*
import simplejson as json
print(json.loads('"char"'))

--
Regards,
Murtuza Zabuawala
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git 
a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py
 
b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py
index ac9a103..225e694 100644
--- 
a/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py
+++ 
b/web/pgadmin/browser/server_groups/servers/databases/schemas/tables/column/__init__.py
@@ -490,6 +490,15 @@ class ColumnsView(PGChildNodeView, DataTypeReader):
 after length/precision so we will set flag for
 sql template
 """
+
+# We need to add this exceptional case for manually adding " in type
+# in json.loads('"char"') is valid json hence it
+# converts '"char"' -> 'char' as string but if we
+# send the same in collection json.loads() handles it properly in
+# Table & Type nodes, This handling handling is Column node specific
+if type == 'char':
+type = '"char"'
+
 if '[]' in type:
 type = type.replace('[]', '')
 self.hasSqrBracket = True
@@ -710,7 +719,7 @@ class ColumnsView(PGChildNodeView, DataTypeReader):
 for k, v in request.args.items():
 try:
 data[k] = json.loads(v, encoding='utf-8')
-except ValueError:
+except (ValueError, TypeError, KeyError):
 data[k] = v
 
 # Adding parent into data dict, will be using it while creating sql

-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] Acceptance Tests against a browser (WIP)

[Dave Page]

Hi

On Fri, Feb 3, 2017 at 9:56 PM, Atira Odhner  wrote:
> Hi Dave,
>
> Here is a new patch which includes the following:
> - randomized ports
> - delete the acceptance_test_db database in setup in case a prior run failed
> - fixed size browser window

Definitely getting there :-). A couple of thoughts/questions:

- Now there are 2 tests in there, it's clear that both the Python
server and browser session are restarted for each test. Can this be
avoided? It'll really slow down test execution as more and more are
added.

- We've got a new monster name:
pgadmin.acceptance.tests.sql_template_selection_by_postgres_version_works_feature_test.SQLTemplateSelectionByPostgresVersionWorksFeatureTest
(which on disk is
sql_template_select_by_postgres_version_works_feature_test.py). Names
like that really must be shortened to something more sane and
manageable.

- I'm a little confused by why the tests cannot be run in server mode.
The error says it's because the username/password is unknown -
however, both the pgAdmin and database server usernames and passwords
are in test_config.json.

Thanks!

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] pgAdmin 4 commit: Fix display of graphical query plans for UPDATE/DELET

[Dave Page]

Fix display of graphical query plans for UPDATE/DELETE queries. Fixes #2133

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=4d8683e13ac9988b99fa345a7ddf40dfa2711ffe
Author: Murtuza Zabuawala 

Modified Files
--
web/pgadmin/misc/templates/explain/js/explain.js | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] [pgAdmin4][PATCH] To fix graphical explain for Insert/Update/Delete quries

[Dave Page]

On Mon, Feb 6, 2017 at 10:33 AM, Murtuza Zabuawala
 wrote:
> Hi Dave,
>
> With Same query I am getting proper output same as pgAdmin3.
> Can you hard refresh browser & try again?

That's really weird - it's working now. It was a single-file change,
and when I tested it went from failing to display the node properly at
all, to displaying it with the incorrect label, so it must have loaded
an update.

Oh well - it works now, so patch applied.

Thanks!

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] pgAdmin 4 commit: Fix handling of "char" columns. Fixes #2152

[Dave Page]

Fix handling of "char" columns. Fixes #2152

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=c96da34c28aa6c705ae005d542022e48b0c6a65a
Author: Murtuza Zabuawala 

Modified Files
--
.../servers/databases/schemas/tables/column/__init__.py   | 11 ++-
1 file changed, 10 insertions(+), 1 deletion(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] [pgAdmin4][PATCH] To handle "char" type in column node

[Dave Page]

Thanks - applied.

On Mon, Feb 6, 2017 at 10:20 AM, Murtuza Zabuawala
 wrote:
> Hi,
>
> PFA minor patch to handle "char" type while creating columns in Column node.
>
> Issue: simplejson parse '"char"' as valid json, hence when we perform
> simplejson.loads(request.args) operation '"char"' gets converted to 'char'
> I had to handle it manually as of now, I did not find any other workaround
> to handle this issue, I went over several SO question most of them suggested
> manual handling.
> This behavior is only in Column node, table & type nodes are not affectted
> by .loads() method as we send data as column collection.
> RM#2152
>
> Snippet of issue:
> import simplejson as json
> print(json.loads('"char"'))
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
> --
> Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgadmin-hackers
>



-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] pgAdmin 4 commit: Compatibility fixes for newer versions of Jinja2 (e.g

[Dave Page]

Compatibility fixes for newer versions of Jinja2 (e.g. 2.9.5+). Fixes #2156

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=ec930770407a23e4ca48cc55a42f9591732c5f56
Author: Murtuza Zabuawala 

Modified Files
--
web/pgadmin/browser/templates/browser/js/browser.js | 6 +++---
web/pgadmin/templates/base.html | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] Patch from RM1983 [pgAdmin4]

[Harshal Dhumal]

Hi,

Please find attached patch for RM 1983.

This issue only occurs when database encoding is other than utf-8

Also other issue was when we use connection of database with encoding other
than utf-8 to retrieve data from cluster table/s which has encoding utf-8
(e.g. pg_database) then data was not decoded properly.



-- 
*Harshal Dhumal*
*Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/web/pgadmin/browser/server_groups/servers/databases/__init__.py b/web/pgadmin/browser/server_groups/servers/databases/__init__.py
index 88360dd..df325e1 100644
--- a/web/pgadmin/browser/server_groups/servers/databases/__init__.py
+++ b/web/pgadmin/browser/server_groups/servers/databases/__init__.py
@@ -9,6 +9,7 @@
 
 """Implements the Database Node"""
 
+import sys
 import simplejson as json
 import re
 from functools import wraps
@@ -568,6 +569,24 @@ class DatabaseView(PGChildNodeView):
 )
 
 data['old_name'] = (rset['rows'][0])['name']
+
+# pg_database is cluster level table (with encoding utf-8)
+# which is common to all databases under that cluster.
+# So if connection of database with encoding other than utf-8 is
+# used to query data from pg_database then manually decode
+# database name to get correct one.
+
+if self.conn.py_encoding != 'utf_8':
+if sys.version_info < (3,):
+data['old_name'] = data['old_name']\
+.decode('utf-8')\
+.encode(self.conn.py_encoding)\
+.decode('utf-8')
+else:
+data['old_name'] = bytes(
+data['old_name'], self.conn.py_encoding
+).decode('utf-8')
+
 if 'name' not in data:
 data['name'] = data['old_name']
 
diff --git a/web/pgadmin/browser/utils.py b/web/pgadmin/browser/utils.py
index 0dc2d01..2224720 100644
--- a/web/pgadmin/browser/utils.py
+++ b/web/pgadmin/browser/utils.py
@@ -11,6 +11,7 @@
 
 from abc import abstractmethod
 
+import sys
 import flask
 from flask import render_template, current_app
 from flask_babel import gettext
@@ -368,7 +369,24 @@ class PGChildNodeView(NodeView):
 current_app.logger.error(result)
 
 for row in result['rows']:
+
+# pg_database is cluster level table (with encoding utf-8)
+# which is common to all databases under that cluster.
+# So if connection of database with encoding other than utf-8 is
+# used to query data from pg_database then manually decode
+# dependencies name to get correct one.
 ref_name = row['refname']
+if conn.py_encoding != 'utf_8':
+if sys.version_info < (3,):
+ref_name = ref_name\
+.decode('utf-8')\
+.encode(conn.py_encoding)\
+.decode('utf-8')
+else:
+ref_name = bytes(
+ref_name, conn.py_encoding
+).decode('utf-8')
+
 dep_str = row['deptype']
 dep_type = ''
 
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 4cafeb2..b442846 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -26,7 +26,7 @@ from flask import g, current_app, session
 from flask_babel import gettext
 from flask_security import current_user
 from pgadmin.utils.crypto import decrypt
-from psycopg2.extensions import adapt
+from psycopg2.extensions import adapt, encodings
 
 import config
 from pgadmin.model import Server, User
@@ -74,6 +74,7 @@ psycopg2.extensions.register_type(
 'NUMERIC_RANGE_TEXT', psycopg2.STRING)
 )
 
+
 def register_string_typecasters(connection):
 """
 Casts various types to string, resolving issues with out of
@@ -94,6 +95,30 @@ def register_string_typecasters(connection):
 new_type = psycopg2.extensions.new_type(oids, 'RETURN_STRING', return_as_string)
 psycopg2.extensions.register_type(new_type)
 
+# In python3 when database encoding is other than utf-8 and client encoding
+# is set to UNICODE then we need to map data from database encoding
+# to utf-8.
+# This is required because when client encoding is set to UNICODE then
+# psycopg assumes database encoding utf-8 and not the actual encoding.
+# Not sure whether it's bug or feature in psycopg for python3.
+
+if sys.version_info >= (3,) and connection.encoding != 'UTF8':
+def return_as_unicode(value, cursor):
+if value is None:
+return None
+# Treat value as byte sequence of database encoding and then decode

Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]

[Dave Page]

Hi

On Mon, Feb 6, 2017 at 12:57 PM, Harshal Dhumal
 wrote:
> Hi,
>
> Please find attached patch for RM 1983.
>
> This issue only occurs when database encoding is other than utf-8
>
> Also other issue was when we use connection of database with encoding other
> than utf-8 to retrieve data from cluster table/s which has encoding utf-8
> (e.g. pg_database) then data was not decoded properly.

The code makes an assumption that pg_database is always utf-8 encoded.
I don't believe that is correct - I believe it's the encoding used in
the database from which the new database was created. The general
advice is that users should avoid using non-7bit ASCII characters in
shared catalogs, e.g. databases and comments etc.

See 
https://www.postgresql.org/message-id/flat/20160216163833.GF31273%40momjian.us#20160216163833.gf31...@momjian.us
for more info for example.

Did pgAdmin 3 just assume it was UTF-8? I suspect it did - and that
just happened to work in most cases.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

Re: [pgadmin-hackers] Acceptance Tests against a browser (WIP)

[Atira Odhner]

I agree that we should rename the test. We've renamed it to
"template_selection_feature_test".
Your other suggestions are captured in our backlog as future improvements.
We definitely can and should do those things but I think it would be
valuable to go ahead and get this suite in and give other devs a chance to
use and iterate on this work.

Thanks,

Tira & George

On Mon, Feb 6, 2017 at 5:32 AM, Dave Page  wrote:

> Hi
>
> On Fri, Feb 3, 2017 at 9:56 PM, Atira Odhner  wrote:
> > Hi Dave,
> >
> > Here is a new patch which includes the following:
> > - randomized ports
> > - delete the acceptance_test_db database in setup in case a prior run
> failed
> > - fixed size browser window
>
> Definitely getting there :-). A couple of thoughts/questions:
>
> - Now there are 2 tests in there, it's clear that both the Python
> server and browser session are restarted for each test. Can this be
> avoided? It'll really slow down test execution as more and more are
> added.
>
> - We've got a new monster name:
> pgadmin.acceptance.tests.sql_template_selection_by_postgres_version_works_
> feature_test.SQLTemplateSelectionByPostgresVersionWorksFeatureTest
> (which on disk is
> sql_template_select_by_postgres_version_works_feature_test.py). Names
> like that really must be shortened to something more sane and
> manageable.
>
> - I'm a little confused by why the tests cannot be run in server mode.
> The error says it's because the username/password is unknown -
> however, both the pgAdmin and database server usernames and passwords
> are in test_config.json.
>
> Thanks!
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/requirements_py2.txt b/requirements_py2.txt
index 4fb05891..998cdabf 100644
--- a/requirements_py2.txt
+++ b/requirements_py2.txt
@@ -36,6 +36,7 @@ testscenarios==0.5.0
 testtools==2.0.0
 traceback2==1.4.0
 unittest2==1.1.0
+selenium==3.0.2
 Werkzeug==0.9.6
 WTForms==2.0.2
 sqlparse==0.1.19
diff --git a/requirements_py3.txt b/requirements_py3.txt
index c4490f52..2239de63 100644
--- a/requirements_py3.txt
+++ b/requirements_py3.txt
@@ -35,6 +35,7 @@ testscenarios==0.5.0
 testtools==2.0.0
 traceback2==1.4.0
 unittest2==1.1.0
+selenium==3.0.2
 Werkzeug==0.9.6
 WTForms==2.0.2
 sqlparse==0.1.19
diff --git a/web/pgAdmin4.py b/web/pgAdmin4.py
index 68848c00..95e675b7 100644
--- a/web/pgAdmin4.py
+++ b/web/pgAdmin4.py
@@ -59,6 +59,12 @@ if 'PGADMIN_PORT' in globals():
  globals()['PGADMIN_PORT'])
 server_port = int(globals()['PGADMIN_PORT'])
 PGADMIN_RUNTIME = True
+elif 'PGADMIN_PORT' in os.environ:
+port = os.environ['PGADMIN_PORT']
+app.logger.debug(
+'Not running under the desktop runtime, port: %s',
+port)
+server_port = int(port)
 else:
 app.logger.debug(
 'Not running under the desktop runtime, port: %s',
diff --git a/web/pgadmin/acceptance/__init__.py 
b/web/pgadmin/acceptance/__init__.py
new file mode 100644
index ..e69de29b
diff --git a/web/pgadmin/acceptance/tests/__init__.py 
b/web/pgadmin/acceptance/tests/__init__.py
new file mode 100644
index ..e69de29b
diff --git a/web/pgadmin/acceptance/tests/connect_to_server_feature_test.py 
b/web/pgadmin/acceptance/tests/connect_to_server_feature_test.py
new file mode 100644
index ..b54696f6
--- /dev/null
+++ b/web/pgadmin/acceptance/tests/connect_to_server_feature_test.py
@@ -0,0 +1,73 @@
+#
+#
+# pgAdmin 4 - PostgreSQL Tools
+#
+# Copyright (C) 2013 - 2017, The pgAdmin Development Team
+# This software is released under the PostgreSQL Licence
+#
+##
+
+import time
+
+from selenium import webdriver
+from selenium.common.exceptions import NoSuchElementException
+from selenium.webdriver import ActionChains
+
+from pgadmin.utils.route import BaseTestGenerator
+
+import subprocess
+import os
+import signal
+import config as app_config
+from regression import test_utils
+from regression.utils.app_starter import AppStarter
+from regression.utils.pgadmin_page import PgadminPage
+
+
+class ConnectsToServerFeatureTest(BaseTestGenerator):
+"""
+Tests that a database connection can be created from the UI
+"""
+
+def setUp(self):
+if app_config.SERVER_MODE:
+self.skipTest("Currently, config is set to start pgadmin in server 
mode. "
+  "This test doesn't know username and password so 
doesn't work in server mode")
+
+driver = webdriver.Chrome()
+self.app_starter = AppStarter(driver, app_config)
+self.page = PgadminPage(driver, app_config)
+
+self.app_starter.start_app()
+self.page.wait_for_app()
+
+
+def runTest(self):
+self.assertEqual(app_config.APP_NAME, self.page.driver.title)
+self.page.wait_for_spinner_to_disappear()
+
+ 

Re: [pgadmin-hackers] Patch from RM1983 [pgAdmin4]

[Harshal Dhumal]

Hi Dave,

-- 
*Harshal Dhumal*
*Software Engineer*

EnterpriseDB India: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Feb 6, 2017 at 6:48 PM, Dave Page  wrote:

> Hi
>
> On Mon, Feb 6, 2017 at 12:57 PM, Harshal Dhumal
>  wrote:
> > Hi,
> >
> > Please find attached patch for RM 1983.
> >
> > This issue only occurs when database encoding is other than utf-8
> >
> > Also other issue was when we use connection of database with encoding
> other
> > than utf-8 to retrieve data from cluster table/s which has encoding utf-8
> > (e.g. pg_database) then data was not decoded properly.
>
> The code makes an assumption that pg_database is always utf-8 encoded.
> I don't believe that is correct - I believe it's the encoding used in
> the database from which the new database was created. The general
> advice is that users should avoid using non-7bit ASCII characters in
> shared catalogs, e.g. databases and comments etc.
>

Ok.

Let me split this into two issues:
i) RM1983 for which I have attached updated patch. (basically I removed
changes related to decode data retried from pg_database when connection
encoding is other than utf-8)
ii) Support to allow user to use non-&bit ASCII characters in shared
catalogs with the help of pgAdmin4.

Regarding your statement about pg_database *"I believe it's the encoding
used in the database from which the new database was created."**. *I found
it little-bit confusing for me (correct me if i'm wrong); As mentioned here
 there
is only one copy of pg_database per cluster. So I assume pg_database is
created when we initialize database cluster and not when we create new
database.



>
> See https://www.postgresql.org/message-id/flat/20160216163833.GF31273%
> 40momjian.us#20160216163833.gf31...@momjian.us
> for more info for example.
>
> Did pgAdmin 3 just assume it was UTF-8? I suspect it did - and that
> just happened to work in most cases.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
diff --git a/web/pgadmin/utils/driver/psycopg2/__init__.py b/web/pgadmin/utils/driver/psycopg2/__init__.py
index 4cafeb2..4e460a6 100644
--- a/web/pgadmin/utils/driver/psycopg2/__init__.py
+++ b/web/pgadmin/utils/driver/psycopg2/__init__.py
@@ -26,7 +26,7 @@ from flask import g, current_app, session
 from flask_babel import gettext
 from flask_security import current_user
 from pgadmin.utils.crypto import decrypt
-from psycopg2.extensions import adapt
+from psycopg2.extensions import adapt, encodings
 
 import config
 from pgadmin.model import Server, User
@@ -74,6 +74,7 @@ psycopg2.extensions.register_type(
 'NUMERIC_RANGE_TEXT', psycopg2.STRING)
 )
 
+
 def register_string_typecasters(connection):
 """
 Casts various types to string, resolving issues with out of
@@ -94,6 +95,30 @@ def register_string_typecasters(connection):
 new_type = psycopg2.extensions.new_type(oids, 'RETURN_STRING', return_as_string)
 psycopg2.extensions.register_type(new_type)
 
+# In python3 when database encoding is other than utf-8 and client encoding
+# is set to UNICODE then we need to map data from database encoding
+# to utf-8.
+# This is required because when client encoding is set to UNICODE then
+# psycopg assumes database encoding utf-8 and not the actual encoding.
+# Not sure whether it's bug or feature in psycopg for python3.
+
+if sys.version_info >= (3,) and connection.encoding != 'UTF8':
+def return_as_unicode(value, cursor):
+if value is None:
+return None
+# Treat value as byte sequence of database encoding and then decode
+# it as utf-8 to get correct unicode value.
+return bytes(
+value, encodings[cursor.connection.encoding]
+).decode('utf-8')
+
+unicode_type = psycopg2.extensions.new_type(
+(19, 18, 25, 1042, 1043, 0),
+'UNICODE', return_as_unicode)
+
+psycopg2.extensions.register_type(unicode_type)
+
+
 class Connection(BaseConnection):
 """
 class Connection(object)
@@ -568,11 +593,17 @@ WHERE
 query: SQL query to run.
 params: Extra parameters
 """
+
+if sys.version_info < (3,):
+if type(query) == unicode:
+query = query.encode('utf-8')
+else:
+query = query.encode('utf-8')
+
 cur.execute(query, params)
 if self.async == 1:
 self._wait(cur.connection)
 
-
 def execute_on_server_as_csv(self, query, params=None, formatted_exception_msg=False, records=2000):
 status, cur = self.__cursor(server_cursor=True)
 self.row_count = 0
@@ -581,11 +612,14 @@ WHERE
 return False, str(cur)
 query_id = random.randint(1, 999)
 
+if sys.version_info < (3,) an

[pgadmin-hackers] pgAdmin 4 commit: Bump version for release.

[Dave Page]

Bump version for release.

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=5abd62389decefdedd7dc837602cb019ccca44dc

Modified Files
--
web/config.py | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers

[pgadmin-hackers] pgAdmin 4 commit: Update translation files.

[Dave Page]

Update translation files.

Branch
--
master

Details
---
http://git.postgresql.org/gitweb?p=pgadmin4.git;a=commitdiff;h=2dc0eab057d797a1fecb9947239be5638b442c5d

Modified Files
--
web/pgadmin/messages.pot   | 3118 +-
.../translations/fr/LC_MESSAGES/messages.po| 3398 
.../translations/zh/LC_MESSAGES/messages.mo|  Bin 100980 -> 101039 bytes
.../translations/zh/LC_MESSAGES/messages.po| 3335 ++-
4 files changed, 5216 insertions(+), 4635 deletions(-)


-- 
Sent via pgadmin-hackers mailing list (pgadmin-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgadmin-hackers