Re: [Geoserver-users] WFS will not read the same date that it stores
There’s also a problem handling date/times. See the following Insert
transaction that populates an Oracle column of type TIMESTAMP.
http://www.opengis.net/wfs";
xmlns:gml="http://www.opengis.net/gml";
xmlns:ogc="http://www.opengis.net/ogc";
version="1.0.0"
service="WFS">
2016-01-04T00:00:00
This attempts to store a date/time of 1/4/2016 00:00:00. What actually gets
stored in Oracle is 1/3/2016 07:00:00 PM.
I have fixed both the date and date/time problems in my development environment
as follows:
See the code below in XsDateTimeFormat, unchanged from the github download of
GeoTools:
TimeZone tz;
if (parseTime) {
tz = TimeZone.getTimeZone(digits.toString());
} else {
// there's no meaning for timezone if not parsing time
// http://en.wikipedia.org/wiki/ISO_8601
tz = TimeZone.getTimeZone("GMT");
}
Here are my changes to that block of code:
TimeZone tz;
// Apply timezone arithmetic when z+/- has been used.
if (parseTime && digits.toString().length() != 3) {
// Set the timezone based on z+/- arithmetic in the time.
tz = TimeZone.getTimeZone(digits.toString());
} else {
// Set the timezone for times or dates based on the default.
// Note that timezone does have meaning for dates because
// it facilitates handling the value as a date/time of
// midnight that day and timezone changes can adjust the
// value of the date itself.
tz = TimeZone.getDefault();
}
The differences in my code is that it uses the GMT+/- offset timezone only if
z+/- arithmetic has been done in the time string. In all other cases it uses
the default timezone of the JVM.
From: andrea.a...@gmail.com [mailto:andrea.a...@gmail.com] On Behalf Of Andrea
Aime
Sent: Thursday, October 13, 2016 2:55 PM
To: Ian Turton
Cc: Walter Stovall ; geoserver-users
Subject: Re: [Geoserver-users] WFS will not read the same date that it stores
Ian, going by memory and answering from my tablet... didn't someone contribute
a patch to make
dates timezone independent? With a system variable to set that?
Cheers
Andrea
On Thu, Oct 13, 2016 at 8:22 PM, Ian Turton
mailto:ijtur...@gmail.com>> wrote:
That sounds like a regression to me, when I'm back at my desk, I'll dig out the
previous tests and check how that slipped in.
Can you raise a bug in the meantime so I don't forget.
Ian
On 13 Oct 2016 16:28, "Walter Stovall"
mailto:walter.stov...@byers.com>> wrote:
Thanks for the advice. Unfortunately following that path does not work for me.
I still end up with dates that are one-behind.
This was discovered with a more complicated arrangement, but I have this
problem in my simple test environment where my geoserver is deployed to a local
tomcat 8 on my Windows workstation. This is a computer setup for Eastern
Standard Daylight time (which is an offset of -05:00).
I browse to my http://localhost:8080/geoserver/web webapp and run the Demo page
where I paste in the WFS-T Insert request I showed earlier that inserts a date
of 01/04/2016. Then I see a new row has showed up in my back-end Oracle table
where geoserver is storing these features with the OracleNGDataStore.
Selecting the date from that row, I get 01/03/2016. If I drill down by asking
Oracle to show the time too I’ll get 03-jan-2016 19:00:00. So obviously
“somebody” is applying the -05:00 offset by the time the data is given to the
Oracle driver. fwiw the Oracle dbtimezone is -04:00 but that hardly matters
because Oracle expects to be given the data in the “correct” timezone and just
stores what you give it.
I’ve already discovered by watching the geoserver log that Oracle is receiving
the wrong value.
To find out more I build an eclipse workspace that includes geoserver and
geotools and attach to the running tomcat.
So first I break in the debugger at SimpleFeatureImpl which is getting hit by
the SAX parsing invoked by WfsXmlReader. The values object array there has a
date of “2016-01-03” (which is the toString output of Gregorian$Date with a
fastTime of 145186560). This is ultimately going to become Jan 3 19:00
stored in Oracle. So I know things are bad already.
Backtracking to see where things went wrong I back all the way up to where the
Jan 4 date is getting parsed out of the XML for my WFS-T insert request. I
stop at XsDateTimeFormat.parseObject(String, ParsePosition, boolean) and in
there I see where there’s an assumption that the timezone will ALWAYS be GMT…
if (parseTime) {
tz = TimeZone.getTimeZone(digits.toString());
} else {
// there's no meaning for timezone if not parsing time
// http://en.wikipedia.org/wiki/ISO_8601
Re: [Geoserver-users] WFS will not read the same date that it stores
I am pretty sure that was me. But I'll check tomorrow.
Ian
On 13 Oct 2016 19:54, "Andrea Aime" wrote:
> Ian, going by memory and answering from my tablet... didn't someone
> contribute a patch to make
> dates timezone independent? With a system variable to set that?
>
> Cheers
> Andrea
>
> On Thu, Oct 13, 2016 at 8:22 PM, Ian Turton wrote:
>
>> That sounds like a regression to me, when I'm back at my desk, I'll dig
>> out the previous tests and check how that slipped in.
>>
>> Can you raise a bug in the meantime so I don't forget.
>>
>> Ian
>>
>> On 13 Oct 2016 16:28, "Walter Stovall" wrote:
>>
>>> Thanks for the advice. Unfortunately following that path does not work
>>> for me. I still end up with dates that are one-behind.
>>>
>>>
>>>
>>> This was discovered with a more complicated arrangement, but I have this
>>> problem in my simple test environment where my geoserver is deployed to a
>>> local tomcat 8 on my Windows workstation. This is a computer setup for
>>> Eastern Standard Daylight time (which is an offset of -05:00).
>>>
>>>
>>>
>>> I browse to my http://localhost:8080/geoserver/web webapp and run the
>>> Demo page where I paste in the WFS-T Insert request I showed earlier that
>>> inserts a date of 01/04/2016. Then I see a new row has showed up in my
>>> back-end Oracle table where geoserver is storing these features with the
>>> OracleNGDataStore. Selecting the date from that row, I get 01/03/2016. If
>>> I drill down by asking Oracle to show the time too I’ll get 03-jan-2016
>>> 19:00:00. So obviously “somebody” is applying the -05:00 offset by the
>>> time the data is given to the Oracle driver. fwiw the Oracle dbtimezone is
>>> -04:00 but that hardly matters because Oracle expects to be given the data
>>> in the “correct” timezone and just stores what you give it.
>>>
>>>
>>>
>>> I’ve already discovered by watching the geoserver log that Oracle is
>>> receiving the wrong value.
>>>
>>>
>>>
>>> To find out more I build an eclipse workspace that includes geoserver
>>> and geotools and attach to the running tomcat.
>>>
>>> So first I break in the debugger at SimpleFeatureImpl which is getting
>>> hit by the SAX parsing invoked by WfsXmlReader. The *values* object
>>> array there has a date of “2016-01-03” (which is the toString output of
>>> Gregorian$Date with a fastTime of 145186560). This is ultimately going
>>> to become Jan 3 19:00 stored in Oracle. So I know things are bad already.
>>>
>>>
>>>
>>> Backtracking to see where things went wrong I back all the way up to
>>> where the Jan 4 date is getting parsed out of the XML for my WFS-T insert
>>> request. I stop at XsDateTimeFormat.parseObject(String, ParsePosition,
>>> boolean) and in there I see where there’s an assumption that the timezone
>>> will ALWAYS be GMT…
>>>
>>>
>>>
>>> *if* (parseTime) {
>>>
>>> tz = TimeZone.*getTimeZone*(digits.toString());
>>>
>>> } *else* {
>>>
>>> // there's no meaning for timezone if not parsing time
>>>
>>> // http://en.wikipedia.org/wiki/ISO_8601
>>>
>>> tz = TimeZone.*getTimeZone*("GMT");
>>>
>>> }
>>>
>>> Calendar cal = Calendar.*getInstance*(tz);
>>>
>>>
>>>
>>> So regardless of environment variables etc. I know that at this stage my
>>> incoming date is always represented as Jan 4 2016 GMT. It would appear
>>> that later this gets represented as EDT (my timezone) and becomes Jan 3
>>> 19:00.
>>>
>>>
>>>
>>> So I patched my code just now to call TimeZone.getDefault() instead of
>>> using “GMT” specifically. Now what gets stored in my database is correct,
>>> Jan 4 00:00.
>>>
>>>
>>>
>>> As to whether this specifically is a fix and whether similar problems
>>> wait for me with datetime vs. just date, I’m unclear at this point.
>>>
>>>
>>>
>>> What are your thoughts about this? Do I have a way to make it work
>>> without a code change?
>>>
>>>
>>>
>>> Thanks for your help Ian!
>>>
>>>
>>>
>>> Regards Walter Stovall
>
Re: [Geoserver-users] WFS will not read the same date that it stores
Ian, going by memory and answering from my tablet... didn't someone
contribute a patch to make
dates timezone independent? With a system variable to set that?
Cheers
Andrea
On Thu, Oct 13, 2016 at 8:22 PM, Ian Turton wrote:
> That sounds like a regression to me, when I'm back at my desk, I'll dig
> out the previous tests and check how that slipped in.
>
> Can you raise a bug in the meantime so I don't forget.
>
> Ian
>
> On 13 Oct 2016 16:28, "Walter Stovall" wrote:
>
>> Thanks for the advice. Unfortunately following that path does not work
>> for me. I still end up with dates that are one-behind.
>>
>>
>>
>> This was discovered with a more complicated arrangement, but I have this
>> problem in my simple test environment where my geoserver is deployed to a
>> local tomcat 8 on my Windows workstation. This is a computer setup for
>> Eastern Standard Daylight time (which is an offset of -05:00).
>>
>>
>>
>> I browse to my http://localhost:8080/geoserver/web webapp and run the
>> Demo page where I paste in the WFS-T Insert request I showed earlier that
>> inserts a date of 01/04/2016. Then I see a new row has showed up in my
>> back-end Oracle table where geoserver is storing these features with the
>> OracleNGDataStore. Selecting the date from that row, I get 01/03/2016. If
>> I drill down by asking Oracle to show the time too I’ll get 03-jan-2016
>> 19:00:00. So obviously “somebody” is applying the -05:00 offset by the
>> time the data is given to the Oracle driver. fwiw the Oracle dbtimezone is
>> -04:00 but that hardly matters because Oracle expects to be given the data
>> in the “correct” timezone and just stores what you give it.
>>
>>
>>
>> I’ve already discovered by watching the geoserver log that Oracle is
>> receiving the wrong value.
>>
>>
>>
>> To find out more I build an eclipse workspace that includes geoserver and
>> geotools and attach to the running tomcat.
>>
>> So first I break in the debugger at SimpleFeatureImpl which is getting
>> hit by the SAX parsing invoked by WfsXmlReader. The *values* object
>> array there has a date of “2016-01-03” (which is the toString output of
>> Gregorian$Date with a fastTime of 145186560). This is ultimately going
>> to become Jan 3 19:00 stored in Oracle. So I know things are bad already.
>>
>>
>>
>> Backtracking to see where things went wrong I back all the way up to
>> where the Jan 4 date is getting parsed out of the XML for my WFS-T insert
>> request. I stop at XsDateTimeFormat.parseObject(String, ParsePosition,
>> boolean) and in there I see where there’s an assumption that the timezone
>> will ALWAYS be GMT…
>>
>>
>>
>> *if* (parseTime) {
>>
>> tz = TimeZone.*getTimeZone*(digits.toString());
>>
>> } *else* {
>>
>> // there's no meaning for timezone if not parsing time
>>
>> // http://en.wikipedia.org/wiki/ISO_8601
>>
>> tz = TimeZone.*getTimeZone*("GMT");
>>
>> }
>>
>> Calendar cal = Calendar.*getInstance*(tz);
>>
>>
>>
>> So regardless of environment variables etc. I know that at this stage my
>> incoming date is always represented as Jan 4 2016 GMT. It would appear
>> that later this gets represented as EDT (my timezone) and becomes Jan 3
>> 19:00.
>>
>>
>>
>> So I patched my code just now to call TimeZone.getDefault() instead of
>> using “GMT” specifically. Now what gets stored in my database is correct,
>> Jan 4 00:00.
>>
>>
>>
>> As to whether this specifically is a fix and whether similar problems
>> wait for me with datetime vs. just date, I’m unclear at this point.
>>
>>
>>
>> What are your thoughts about this? Do I have a way to make it work
>> without a code change?
>>
>>
>>
>> Thanks for your help Ian!
>>
>>
>>
>> Regards Walter Stovall
>>
>>
>>
>> *From:* Ian Turton [mailto:ijtur...@gmail.com]
>> *Sent:* Thursday, October 13, 2016 9:54 AM
>> *To:* Walter Stovall
>> *Cc:* geoserver-users@lists.sourceforge.net
>> *Subject:* Re: [Geoserver-users] WFS will not read the same date that it
>> stores
>>
>>
>>
>> The GeoTools/GeoServer data handler should use the timezone of the server
>> (so if that is set to GMT then you need to convert before sending). This
>> allows for servers where clients can be in multiple timezones
Re: [Geoserver-users] WFS will not read the same date that it stores
That sounds like a regression to me, when I'm back at my desk, I'll dig out
the previous tests and check how that slipped in.
Can you raise a bug in the meantime so I don't forget.
Ian
On 13 Oct 2016 16:28, "Walter Stovall" wrote:
> Thanks for the advice. Unfortunately following that path does not work for
> me. I still end up with dates that are one-behind.
>
>
>
> This was discovered with a more complicated arrangement, but I have this
> problem in my simple test environment where my geoserver is deployed to a
> local tomcat 8 on my Windows workstation. This is a computer setup for
> Eastern Standard Daylight time (which is an offset of -05:00).
>
>
>
> I browse to my http://localhost:8080/geoserver/web webapp and run the
> Demo page where I paste in the WFS-T Insert request I showed earlier that
> inserts a date of 01/04/2016. Then I see a new row has showed up in my
> back-end Oracle table where geoserver is storing these features with the
> OracleNGDataStore. Selecting the date from that row, I get 01/03/2016. If
> I drill down by asking Oracle to show the time too I’ll get 03-jan-2016
> 19:00:00. So obviously “somebody” is applying the -05:00 offset by the
> time the data is given to the Oracle driver. fwiw the Oracle dbtimezone is
> -04:00 but that hardly matters because Oracle expects to be given the data
> in the “correct” timezone and just stores what you give it.
>
>
>
> I’ve already discovered by watching the geoserver log that Oracle is
> receiving the wrong value.
>
>
>
> To find out more I build an eclipse workspace that includes geoserver and
> geotools and attach to the running tomcat.
>
> So first I break in the debugger at SimpleFeatureImpl which is getting hit
> by the SAX parsing invoked by WfsXmlReader. The *values* object array
> there has a date of “2016-01-03” (which is the toString output of
> Gregorian$Date with a fastTime of 145186560). This is ultimately going
> to become Jan 3 19:00 stored in Oracle. So I know things are bad already.
>
>
>
> Backtracking to see where things went wrong I back all the way up to where
> the Jan 4 date is getting parsed out of the XML for my WFS-T insert
> request. I stop at XsDateTimeFormat.parseObject(String, ParsePosition,
> boolean) and in there I see where there’s an assumption that the timezone
> will ALWAYS be GMT…
>
>
>
> *if* (parseTime) {
>
> tz = TimeZone.*getTimeZone*(digits.toString());
>
> } *else* {
>
> // there's no meaning for timezone if not parsing time
>
> // http://en.wikipedia.org/wiki/ISO_8601
>
> tz = TimeZone.*getTimeZone*("GMT");
>
> }
>
> Calendar cal = Calendar.*getInstance*(tz);
>
>
>
> So regardless of environment variables etc. I know that at this stage my
> incoming date is always represented as Jan 4 2016 GMT. It would appear
> that later this gets represented as EDT (my timezone) and becomes Jan 3
> 19:00.
>
>
>
> So I patched my code just now to call TimeZone.getDefault() instead of
> using “GMT” specifically. Now what gets stored in my database is correct,
> Jan 4 00:00.
>
>
>
> As to whether this specifically is a fix and whether similar problems wait
> for me with datetime vs. just date, I’m unclear at this point.
>
>
>
> What are your thoughts about this? Do I have a way to make it work
> without a code change?
>
>
>
> Thanks for your help Ian!
>
>
>
> Regards Walter Stovall
>
>
>
> *From:* Ian Turton [mailto:ijtur...@gmail.com]
> *Sent:* Thursday, October 13, 2016 9:54 AM
> *To:* Walter Stovall
> *Cc:* geoserver-users@lists.sourceforge.net
> *Subject:* Re: [Geoserver-users] WFS will not read the same date that it
> stores
>
>
>
> The GeoTools/GeoServer data handler should use the timezone of the server
> (so if that is set to GMT then you need to convert before sending). This
> allows for servers where clients can be in multiple timezones and need to
> agree a fixed timezone when communicating with the server.
>
>
>
> If you are the only user of the client/server then you should be able to
> set the timezone on the server and client to the same one and it should all
> just work (tm).
>
>
>
> Ian
>
>
>
> On 13 October 2016 at 14:33, Walter Stovall
> wrote:
>
> Using GeoServer2.8 I’m having a very basic problem with WFS-T inserting a
> feature that has date fields. When I do a GetFeature to retrieve the date
> I inserted, I get a value that’s one-day prior to the date I inserted.
> This appears to be related to the GeoTools xsDateTimeFormat class used to
> read the i
Re: [Geoserver-users] WFS will not read the same date that it stores
Thanks for the advice. Unfortunately following that path does not work for me.
I still end up with dates that are one-behind.
This was discovered with a more complicated arrangement, but I have this
problem in my simple test environment where my geoserver is deployed to a local
tomcat 8 on my Windows workstation. This is a computer setup for Eastern
Standard Daylight time (which is an offset of -05:00).
I browse to my http://localhost:8080/geoserver/web webapp and run the Demo page
where I paste in the WFS-T Insert request I showed earlier that inserts a date
of 01/04/2016. Then I see a new row has showed up in my back-end Oracle table
where geoserver is storing these features with the OracleNGDataStore.
Selecting the date from that row, I get 01/03/2016. If I drill down by asking
Oracle to show the time too I’ll get 03-jan-2016 19:00:00. So obviously
“somebody” is applying the -05:00 offset by the time the data is given to the
Oracle driver. fwiw the Oracle dbtimezone is -04:00 but that hardly matters
because Oracle expects to be given the data in the “correct” timezone and just
stores what you give it.
I’ve already discovered by watching the geoserver log that Oracle is receiving
the wrong value.
To find out more I build an eclipse workspace that includes geoserver and
geotools and attach to the running tomcat.
So first I break in the debugger at SimpleFeatureImpl which is getting hit by
the SAX parsing invoked by WfsXmlReader. The values object array there has a
date of “2016-01-03” (which is the toString output of Gregorian$Date with a
fastTime of 145186560). This is ultimately going to become Jan 3 19:00
stored in Oracle. So I know things are bad already.
Backtracking to see where things went wrong I back all the way up to where the
Jan 4 date is getting parsed out of the XML for my WFS-T insert request. I
stop at XsDateTimeFormat.parseObject(String, ParsePosition, boolean) and in
there I see where there’s an assumption that the timezone will ALWAYS be GMT…
if (parseTime) {
tz = TimeZone.getTimeZone(digits.toString());
} else {
// there's no meaning for timezone if not parsing time
// http://en.wikipedia.org/wiki/ISO_8601
tz = TimeZone.getTimeZone("GMT");
}
Calendar cal = Calendar.getInstance(tz);
So regardless of environment variables etc. I know that at this stage my
incoming date is always represented as Jan 4 2016 GMT. It would appear that
later this gets represented as EDT (my timezone) and becomes Jan 3 19:00.
So I patched my code just now to call TimeZone.getDefault() instead of using
“GMT” specifically. Now what gets stored in my database is correct, Jan 4
00:00.
As to whether this specifically is a fix and whether similar problems wait for
me with datetime vs. just date, I’m unclear at this point.
What are your thoughts about this? Do I have a way to make it work without a
code change?
Thanks for your help Ian!
Regards Walter Stovall
From: Ian Turton [mailto:ijtur...@gmail.com]
Sent: Thursday, October 13, 2016 9:54 AM
To: Walter Stovall
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] WFS will not read the same date that it stores
The GeoTools/GeoServer data handler should use the timezone of the server (so
if that is set to GMT then you need to convert before sending). This allows for
servers where clients can be in multiple timezones and need to agree a fixed
timezone when communicating with the server.
If you are the only user of the client/server then you should be able to set
the timezone on the server and client to the same one and it should all just
work (tm).
Ian
On 13 October 2016 at 14:33, Walter Stovall
mailto:walter.stov...@byers.com>> wrote:
Using GeoServer2.8 I’m having a very basic problem with WFS-T inserting a
feature that has date fields. When I do a GetFeature to retrieve the date I
inserted, I get a value that’s one-day prior to the date I inserted. This
appears to be related to the GeoTools xsDateTimeFormat class used to read the
incoming XML on my insert-request and the code’s assumption that the incoming
date is in the GMT timezone. Do clients of WFS have to convert all their dates
to GMT before inserting them??
Here is an example transaction that I use for insert…
http://www.opengis.net/wfs";
xmlns:gml="http://www.opengis.net/gml";
xmlns:ogc="http://www.opengis.net/ogc";
version="1.0.0"
service="WFS">
2016-01-04
Then I read it back with…
http://www.opengis.net/wfs";
xmlns:fdot="http://byers.com/nxwx/solution/fdot";
xmlns:gml="http://www.opengis.net/gml";
xmlns:ogc="http://www.opengis.net/ogc";
version="1.0.0"
service="WFS"
outputFormat="JSON">
In the JSON output I find…
"INSTALLED
Re: [Geoserver-users] WFS will not read the same date that it stores
The GeoTools/GeoServer data handler should use the timezone of the server (so if that is set to GMT then you need to convert before sending). This allows for servers where clients can be in multiple timezones and need to agree a fixed timezone when communicating with the server. If you are the only user of the client/server then you should be able to set the timezone on the server and client to the same one and it should all just work (tm). Ian On 13 October 2016 at 14:33, Walter Stovall wrote: > Using GeoServer2.8 I’m having a very basic problem with WFS-T inserting a > feature that has date fields. When I do a GetFeature to retrieve the date > I inserted, I get a value that’s one-day prior to the date I inserted. > This appears to be related to the GeoTools xsDateTimeFormat class used to > read the incoming XML on my insert-request and the code’s assumption that > the incoming date is in the GMT timezone. Do clients of WFS have to > convert all their dates to GMT before inserting them?? > > > > Here is an example transaction that I use for insert… > > > > > xmlns="http://www.opengis.net/wfs"; > > xmlns:gml="http://www.opengis.net/gml"; > > xmlns:ogc="http://www.opengis.net/ogc"; > > version="1.0.0" > > service="WFS"> > > > > > > 2016-01-04 > > > > > > > > > > Then I read it back with… > > > > > xmlns="http://www.opengis.net/wfs"; > > xmlns:fdot="http://byers.com/nxwx/solution/fdot"; > > xmlns:gml="http://www.opengis.net/gml"; > > xmlns:ogc="http://www.opengis.net/ogc"; > > version="1.0.0" > > service="WFS" > > outputFormat="JSON"> > > > > > > > > > > > > > > > > In the JSON output I find… > > "INSTALLED_DATE":"2016-01-03Z" > > > > So I stored 1/4/2016, I read back 1/3/2016. > > > > My back-end datastore is Oracle. When I select directly from the database > I find the wrong value there too. > > SELECT TO_CHAR(installed_date, 'dd-mon- hh24:mi:ss')… > > > > This gives me… 03-jan-2016 19:00:00. > > > > How do I set things up so a WFS client that stores 1/4/2016 gets 1/4/2016 > back? > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, SlashDot.org! http://sdm.link/slashdot > ___ > Geoserver-users mailing list > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > -- Ian Turton -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
