[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-09-01 Thread Smalyshev
Smalyshev added a comment.

I think as it concerns Blazegraph and RDF, everything that needed to be done is 
done. We support XSD 1.1 now. So I am removing the Wikidata Query parts from it.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Ricordisamoa, thiemowmde, Jc3s5h, Lydia_Pintscher, Denny, Manybubbles, 
daniel, mkroetzsch, Smalyshev, JanZerebecki, Aklapper, jkroll, Wikidata-bugs, 
Jdouglas, aude, Deskana, Malyacko



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-05-19 Thread Jc3s5h
Jc3s5h added a comment.

In https://phabricator.wikimedia.org/T94064#1296543, @daniel wrote:

 @mkroetzsch precise dates for prehistoric times may be useful for 
 astronomical events. These could/should use a different calendar model 
 though, such as https://en.wikipedia.org/wiki/Julian_day (Q14267), see 
 T59704: Support Julian Date (astronomy) 
 https://phabricator.wikimedia.org/T59704


To support dates of prehistoric astronomical events, the issue isn't whether to 
use the Julian day, the proleptic Julian calendar, or the proleptic Gregorian 
calendar. The issue is that the theories of motion of the Solar System, use 
time scales such as Terrestrial Time or Barycentric Coordinate Time. These time 
scales use seconds of equal length, very similar to the seconds produced by 
atomic clocks. But calendars conventionally count actual solar days. Because 
the rate of rotation of the Earth is steadily decreasing **January 3, 10,000 
BC, Julian proleptic calendar modified to observe Terrestrial Time** would be 
approximately **January 1, 10,000, Julian proleptic calendar**.

The actual rotation rate of the Earth is not well known enough to create a 
definitive conversion between calendar dates and Terrestrial Time (and other 
similar timescales) dates. So support of prehistoric astronomical events would 
require support for multiple time scales; one or more for actual observed solar 
days, and others where the length of a day is close to 86,400 atomic seconds.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Jc3s5h
Cc: thiemowmde, Jc3s5h, Lydia_Pintscher, Denny, Manybubbles, daniel, 
mkroetzsch, Smalyshev, JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, 
aude



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-05-19 Thread daniel
daniel added a comment.

@mkroetzsch precise dates for prehistoric times may be useful for astronomical 
events. Thouthe could/should use a different calendar model though, such as 
https://en.wikipedia.org/wiki/Julian_day (Q14267)


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, daniel
Cc: thiemowmde, Jc3s5h, Lydia_Pintscher, Denny, Manybubbles, daniel, 
mkroetzsch, Smalyshev, JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, 
aude



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-05-19 Thread mkroetzsch
mkroetzsch added a comment.

@Jc3s5h You are right that date conversion only makes sense in a certain range. 
I think the software should disallow day-precision dates in prehistoric eras 
(certainly everything before -1). There are no records that could possibly 
justify this precision, and the question of calendar conversion becomes moot. 
Do you think 4713BCE would be enough already, or do you think there could be a 
reason to find more complex algorithms to get calendar support that extends 
further to the past?


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: thiemowmde, Jc3s5h, Lydia_Pintscher, Denny, Manybubbles, daniel, 
mkroetzsch, Smalyshev, JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, 
aude



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-31 Thread mkroetzsch
mkroetzsch added a comment.

@Smalyshev You comment on my Item 1 by referring to BlazeGraph and Virtuoso. 
However, my Item 1 is about reading Wikidata, not about exporting to RDF. Your 
concerns about BlazeGraph compatibility are addressed by my item 2. I hope this 
clarifies this part.

As for the wrong dates, I simply say that we do not know how to fix them, since 
the errors are not sufficiently systematic. At best we can replace one kind of 
error with another kind of error. I agree with you that wrong dates are a bad 
thing, but a bad thing that is beyond our power to fix. We should focus on the 
RDF export and rely on others to do their work, so that everything will run 
smoothly in the end. At the current stage of the RDF work, the issue is of 
relatively minor relevance compared to the problems it is causing elsewhere. 
All of our current RDF exports and several applications that people are using 
suffer from the same errors in Wikidata. We need to fix it at the root, not in 
each consumer.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

 Dates without years should not be allowed by the time datatype


That's fine but they are already there, so I'm not sure how we can say should 
not be allowed there. We have to do something when we encounter them. What is 
something?

 The additional proposal to revert to the dates are just strings view for 
 deep values ignores the original design and documentation, and dismisses the 
 recommendations that Denny and I have been making via email.


It does not dismiss anything. Given current state of the data (invalid dates, 
zero dates, etc.) I do not see how we can faithfully represent the data in the 
deep value as anything else. If you have better idea, please propose. If/when 
we get guarantee that the date in the source value is valid representable 
Gregorian date, we can type it as xsd:dateTime, but before that I don't see how 
we can do that.

 suggest to freeze the RDF-time encoding discussions now until we have 
 established a joint understanding


Establishing understanding is fine, but the code which produces RDF has to 
produce something when it encounters time value. We can not just have all the 
work wait until undefined time where we reach an understanding. So what should 
this code produce now?

 As soon as we export dates to RDF, we are defining their meaning indirectly 
 via the RDF semantics, and this bug report is not the right place for doing 
 this.


We can open another task if needed, though I don't see why this one is 
particularly unsuitable. In any case, I am not proposing anything that has to 
be enshrined as forever standard, and we are not releasing the dump as even 
internal standard, let alone something we promise never to change publicly. But 
we need to have something so that we could have it working and use it for query 
engine work.

 We have to await their report and suggestions before deciding what to do in 
 RDF.


I don't think halting all work on query engine until we reach full consensus on 
this point is realistic. I don't also think that data not including dates is 
really worth any use, even in beta status. So that means we have to export data 
somehow. Even if we know that we may change it before we get out of beta 
status. The question is what that representation would be. I proposed what I 
see as a good solution given the status of affairs //now//. If that's not good, 
fine, I'm completely open to hearing other proposals.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread mkroetzsch
mkroetzsch added a comment.

@Smalyshev

Re halting the work on the query engine/produce code now: The WDTK RDF 
exports are generated based on the original specification. There is no 
technical issue with this and it does not block development to do just this. 
The reason we are in a blocker situation is that you want to move forward with 
an implementation that is different from the RDF model we proposed and that 
goes against our original specification, so that Denny and I are fundamentally 
disagreeing with your design. If you want to return to the original plan, 
please do it and move on. If not, then better wait until Lydia has a conclusion 
for what to do with dates, rather than implementing your point of view without 
consensus. For me, this is a benchmark of whether or not our current discussion 
setup is working.

Here is why I am optimistic that we can align with RDF 1.1 and ISO 8601:2000 
before the query engine would even go live: Basically all calendar-accurate BCE 
dates will be revised and many of them will be changed because of the ongoing 
date review. We can well fix the year zero issue at the same time. Thus we can 
as well work on the hypothesis that dates are in ISO 8601:2000 as originally 
intended. From the feedback we got from the SPARQL group, it seems that this 
would be preferable, if we can make it work technically. The date review is a 
great opportunity to get the whole internal representation back on track.

Re deep value model: the core of the issue is that you propose to represent 
dates as the original string. Denny and I have clarified that we don't find 
this an acceptable representation for dates. As opposed to the XSD 1.0 issue, 
this proposal leads to a completely different structure in RDF and queries. 
There is no upgrade path from this implementation to the one we actually want. 
If we can agree on getting rid of this first, this would be a good start to 
move on. Changing from XSD 1.0 to XSD 1.1 is a minor issue in comparison, and 
one which can be deferred in implementation until we have BlazeGraph support 
for this.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Lydia_Pintscher
Lydia_Pintscher added a comment.

Is there any issue with for now just not including those dates in the RDF 
export? That'd allow @smalyseh to continue working on queries while we figure 
out the rest. It also shouldn't block us in whichever way we go forward later.

And I agree that using year 0 to indicate an unknown year is very bad. We need 
to find a better solution for that usecase.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Lydia_Pintscher
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread mkroetzsch
mkroetzsch added a comment.

 @mkroetzsch I already listed a few of the tools that implement XSD 1.0 style 
 BCE years and I read your answer as to say that you know of no tools that 
 implement XSD 1.1 style BCE years.


Then you misread my answer. Almost all tools that exist today use the 2000 
version of the ISO standard. A prominent example is ECMAScript, and thus all 
JavaScript implementations, and virtually every JavaScript timeline 
implementation. See 
http://www.ecma-international.org/ecma-262/5.1/#sec-15.9.1.15 and the examples 
in the following section to see this. RDF tools are an understandable 
exception, not because people there think one should cling to the old standard, 
but because RDF 1.1 was only standardised in 2014. It is natural that existing 
RDF implementations have more pressing upgrade work to do than to fix BCE date 
handling. I am sure they will all move to the new standard in due course.

It is worth noting that all of the ECMAScript documents use ISO 8601 to refer 
to ISO 8601:2000, just like we did in the Wikidata data model specification. 
It seems that most people are not confused by this.

Having dug up ECMA from the Web, I can now also safely say that JSON exports 
should definitely use ISO 8601:2000 dates.

The new situation therefore is: ISO, W3C, and all JavaScript implementations 
vs. a subset of the developers in the WMDE office. I am very unhappy about the 
amount of my time I have to put into digging up for you what the rest of the 
world is thinking. It's a nice position to put yourself in, asking others to 
find specific arguments against your position and assuming you are right if 
they don't have the time or knowledge to do it. Now I am myself far from being 
an expert in JavaScript or even in all details of SPARQL 1.1, but if I don't 
know something I try to find out before taking part in discussions like this.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread JanZerebecki
JanZerebecki added a comment.

@mkroetzsch I already listed a few of the tools that implement XSD 1.0 style 
BCE years and I read your answer as to say that you know of no tools that 
implement XSD 1.1 style BCE years.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Lydia_Pintscher
Lydia_Pintscher added a comment.

It is the only way I see forward right now that will let you continue working 
on queries quickly. (Anything else needs considerably more 
discussion/investigation as this ticket shows) And I consider it acceptable in 
this case. How many cases are we talking about?


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Lydia_Pintscher
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

 The WDTK RDF exports are generated based on the original specification. There 
 is no technical issue with this and it does not block development to do just 
 this.


If by original specification you mean assumption all data is proleptic 
Gregorian, then it does not match the current data. I.e. if I just make code 
assume that, it will generate a real lot of broken dates which will not be 
interpreted properly by the query engine. In fact, almost all Julian dates will 
be wrong, and many others will be broken too. I'm not sure how useful it would 
be to take this road - why have broken data in our dump?

 If not, then better wait until Lydia has a conclusion for what to do with 
 dates, rather than implementing your point of view without consensus.


I'm not sure how it's better. If any decision is made, we can always change the 
code, but just sitting with our hands folded and doing nothing doesn't look 
like a good idea.

 Re deep value model: the core of the issue is that you propose to represent 
 dates as the original string. Denny and I have clarified that we don't find 
 this an acceptable representation for dates.


OK, but what I still miss is what you consider acceptable that would be able to 
represent current data. If we have date of -02-31 in the data, what you 
propose for the RDF data to contain? What is it's marked as Julian date - what 
should the data contain? What if it is marked with calendar that is neither 
Gregorian nor Julian?

 There is no upgrade path from this implementation to the one we actually want.


Why not? If the format changes, you can update your data pretty easily by 
removing old value nodes/triples and replacing them with new ones. That 
provided somebody would actually use our beta data and get deep enough so it 
would be a problem it time it takes us to make a decision. Which, if that is 
going to take so long time, is yet another argument for not blocking on it.

 Thus we can as well work on the hypothesis that dates are in ISO 8601:2000 as 
 originally intended.


I understand that neither BlazeGraph not Virtuozo do not actually interpret the 
dates as ISO 8601:2000. We need them to understand our dates. I'm not sure how 
you propose to solve this? Or am I mistaken in interpreting Jan's conslusions 
and they are ISO 8601:2000?

@Lydia_Pintscher could you clarify what you mean by those dates? We want to 
represent all dates, I think, so are you proposing to just ignore the triples 
with weird dates? That would mean the data would look as if these dates do 
not exist - which may confuse some queries (e.g person with no date of death is 
considered alive, but it's not the same as somebody having the date of death as 
April 31th 4BCE).


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

@Lydia_Pintscher this means for the dump there will be no difference between 
has property value with date containing year 0 and has no property valye. 
I'm not sure it is a good idea. Having no property value is meaningful (e.g. 
having position start/end date, having death date, having organization 
creation/dissolution date, etc.) so it may lead to wrong query results. We may 
use string or Somevalue, but dropping existing data seems too radical for me.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Lydia_Pintscher
Lydia_Pintscher added a comment.

Yes I am proposing to drop dates that have year set to 0 for now from the RDF 
export. We can communicate that and need to do this anyway as it is really bad 
practice.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, Lydia_Pintscher
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

I'm not sure why it is the only way. Certainly using Somevalue or string or any 
other placeholder value are ways too. They can be worse ways if you see 
arguments against them but why they are not ways at all? Why these ways can not 
be considered? Also, we're not talking only about year 0 - there are many 
non-Gregorian dates there.

Judging by this:
http://milenio.dcc.uchile.cl/sparql?default-graph-uri=query=PREFIX+%3A+%3Chttp%3A%2F%2Fwww.wikidata.org%2Fentity%2F%3E+SELECT+count%28%3Ftime%29+WHERE+%7B+%0D%0A++%3Fx+%3Chttp%3A%2F%2Fwww.wikidata.org%2Fontology%23time%3E+%3Ftime+.+%0D%0A++FILTER+regex%28str%28%3Ftime%29%2C+%22%5E%22%29+.%0D%0A%7D+format=text%2Fhtmltimeout=0debug=on

We have 51 entries with year 0 (if the data there is full and up-to-date) and 
3541 negative dates. Not sure how to count invalid dates but there is a date 
like 1966-02-31^^http://www.w3.org/2001/XMLSchema#date in the DB there which 
doesn't look like a valid one - looks like Virtuoso is more lenient in allowing 
invalid data than BlazeGraph.

We also have 12806 entries marked as Julian.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread mkroetzsch
mkroetzsch added a comment.

@Smalyshev We really want the same thing: move on with minimal disturbance as 
quickly as possible. As you rightly say, the data we generate right now is not 
meant for production use but for testing. We must make sure that our production 
environment will understand dates properly, but it's still some time before 
that. Here is my proposal summed up:

1. Implement RDF export now as if Wikidata would encode all dates in ISO 
8601:2000 (proleptic Gregorian with year 0 encoding -1BCE)
2. Have a switch in the RDF export code that allows us to export to RDF 1.1 or 
to RDF 1.0.

Item 1 will ensure that we can work with the dates as they will most likely be 
when we enter production. With the discovery that all of JavaScript relies on 
ISO 8601:2000, there is not much of a question that we will have this corrected 
in the end. It would be a waste of programming time to work around issues that 
others are already trying to fix as we speak. We can still implement 
reinterpretations of the internal dates when we find that the internal format 
is still broken when we want to release this (I hope this won't happen).

Item 2 is a compromise. It will ensure that we can use BlazeGraph even before 
the xsd:date bug is fixed. Has anyone reported the issue to them yet? It might 
well be that they are quicker fixing this than we are finishing this 
discussion. I am sure they would also like to conform to SPARQL 1.1.

I agree with you that some dates will not be interpreted as intended, but this 
is unavoidable (whatever rule we pick, we will always have some dates that are 
not as intended, already because of the calendar model mix-up). We have to rely 
on the ongoing review to get this fixed. This should not worry us right now, as 
it affects everyone (including actual production uses of Wikidata data from the 
API or JSON dumps).


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

 Item 1 will ensure that we can work with the dates as they will most likely 
 be when we enter production.


I'm not sure what is the basis of this assertion? I didn't see any plans for 
BlazeGraph to move to new standard in the near term, and the same for Virtuoso. 
I will create an issue with Blazegraph but even if/when we convince them to 
move (as this would mean everybody that uses older 1.0 format will not be able 
to receive correct results from RDF anymore, I'm not sure they'd be eager to 
switch) it may take time, and more realistically we'd probably to have to rely 
on our own date handling eventually. However, this is for BlazeGraph, but 
loading the same data into Virtuoso would still produce broken results for any 
BCE date.

More importantly, do we have a commitment that Wikidata data format will be 
changed in the very near future so that all stored dates are actually valid 
proleptic Gregorian? We've initiated this discussion very recently and I didn't 
see any definite resolution yet, much less commitment for the time when the 
data would actually be like this (and how Julian dates will be represented in 
this case). Do you propose that until this happens, our date values in RDF dump 
for every Julian date, evern BCE date, and all dates like -02-31 should be 
unusuable? I'm not sure how that improves anything.

 It would be a waste of programming time to work around issues that others are 
 already trying to fix as we speak.


There's no programming time - current model is already implemented (and Julian 
handling too). Of course, it's not a problem to change it, but for that we need 
to know what we're changing it to and how it would work. Assuming that 
everything is ISO 8601:2000 when in fact it is not does not seem like correct 
way - BlazeGraph would just reject invalid dates (so, not useful for queries), 
and even worse - it would consume current BCE dates not in ISO 8601:2000 but in 
its own understanding of dates, which will make all queries touching 1BCE and 
below invalid, at least until we implement custom date handling. I'm not sure I 
understand why this is the best option, unless we //know// all internal dates 
move to  ISO 8601:2000 very soon.

 I agree with you that some dates will not be interpreted as intended, but 
 this is unavoidable


It is true that there will be invalid dates. But you seem to be proposing just 
ignoring this fact and put them in the data - with full knowledge they are 
invalid and either can not be consumed by the query engine or, even worse, will 
be interpreted incorrectly by the query engine. I am proposing to try and fix 
those that we can so that we allow the query engine to make sense for most of 
them. For others, to just provide string representation and not claim that some 
random assembly of characters that we can not validate is actually xsd:dateTime.

 Your finding of  years in our Virtuoso instance is quite peculiar


Virtuoso seems to be able to import invalid dates. I'm not sure if it's 
actually able to index it (probably not but I can check). However other tools 
can reject them or even fail the whole import.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread mkroetzsch
mkroetzsch added a comment.

@Smalyshev P.S. Your finding of  years in our Virtuoso instance is quite 
peculiar given that this endpoint is based on RDF 1.0 dumps as they are 
currently generated in WDTK using this code: 
https://github.com/Wikidata/Wikidata-Toolkit/blob/a9f676bfbc2df545d386bfa72e5130fa280521a9/wdtk-rdf/src/main/java/org/wikidata/wdtk/rdf/values/TimeValueConverter.java#L112-L117


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-30 Thread Smalyshev
Smalyshev added a comment.

Virtuoso seems to be pretty odd. E.g. this query:

  PREFIX : http://www.wikidata.org/entity/ 
  SELECT ?x  ?time WHERE { 
?x http://www.wikidata.org/ontology#time ?time . 
FILTER (?time  0002-01-01^^xsd:date) 
  } LIMIT 100

returns only a handful of items with year 1 but not any dates with BCE. Same 
when using 0002^^xsd:gYear.  Moreover, this one:

  PREFIX : http://www.wikidata.org/entity/ 
  SELECT count(?time) WHERE { 
?x http://www.wikidata.org/ontology#time ?time . 
FILTER regex(str(?time), ^-) .
FILTER (?time  0001^^http://www.w3.org/2001/XMLSchema#gYear) .
  } LIMIT 100

produces 3541 - i.e., Virtuoso thinks all negative years are actually bigger 
than year 1. No idea what's going on there but I suspect it's not what we want.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-29 Thread JanZerebecki
JanZerebecki added a comment.

@mkroetzsch Do you know of some widely used software that implements XSD 1.1 
handling of BCE dates?

 Dates without years should not be allowed by the time datatype.




 what dates in Wikidata mean


I think the best way forward is to leave the lexical 0 in the year fraction as 
undefined in Wikidata. Yes its used, but AFAIK it was always undefined.

 If this is an important use case, then we should develop a day-of-year 
 datatype that supports this, or suggest the community to use dedicated 
 properties/qualifiers to encode this. However, other datatype extensions 
 would be much more important than this rare case (e.g., units of measurement).


I agree.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-29 Thread mkroetzsch
mkroetzsch added a comment.

@Smalyshev @Lydia_Pintscher Dates without years should not be allowed by the 
time datatype. They are impossible to order, almost impossible to query, and 
they do not have any meaning whatsoever in combination with a preferred 
calendar model. All the arguments @Denny has already given elsewhere for why we 
should unify dates to Proleptic Gregorian internally apply here too. My 
suspicion is that the existing dates of this form are simply a glitch in the 
UI, where users got the impression that dates without years are recognized and 
pressing save silently set the year to zero without them seeing the change in 
meaning. If this is an important use case, then we should develop a day-of-year 
datatype that supports this, or suggest the community to use dedicated 
properties/qualifiers to encode this. However, other datatype extensions would 
be much more important than this rare case (e.g., units of measurement).

The above proposal of @Smalyshev is simply to use RDF 1.0 for export and to 
assume XSD 1.0 (non-ISO) dates to be used in Wikidata. After all the discussion 
here, I am completely baffled by this proposal. It goes against all current 
standards, and against the view of the SPARQL working group. The additional 
proposal to revert to the dates are just strings view for deep values ignores 
the original design and documentation, and dismisses the recommendations that 
Denny and I have been making via email. It seems we have reached an impasse 
here.

**I suggest to freeze the RDF-time encoding discussions now** until we have 
established a joint understanding what dates in Wikidata mean. As soon as we 
export dates to RDF, we are defining their meaning indirectly via the RDF 
semantics, and this bug report is not the right place for doing this.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-29 Thread mkroetzsch
mkroetzsch added a comment.

 @mkroetzsch Do you know of some widely used software that implements XSD 1.1 
 handling of BCE dates?


Many applications that process dates are based on ISO rather than on XSD. 
Java's SimpleDateFormat class, for example, is based on ISO and thus interprets 
year numbers like XSD 1.1. I would assume that most time-processing 
applications, e.g., JavaScript timelines, use the same. Only XSD-based 
implementations tend to have legacy handling. For many RDF tools it is really 
hard to tell without digging into their code (usually they don't document this 
detail, and they use own implementations rather than relying on any XSD 
library). But I think it is fair to assume that ISO has a much larger market 
share, and that XSD 1.0 implementations will be updated at some point.

 I think the best way forward is to leave the lexical 0 in the year fraction 
 as undefined in Wikidata. Yes its used, but AFAIK it was always undefined.


Our original specification of Wikidata said: The calendar model used for 
saving the data is always the proleptic Gregorian calendar according to ISO 
8601. This is the specification that Denny and I support, but there have been 
changes recently. WMDE is currently in the process of reviewing these changes 
to gauge the impact they have had in the data over time, and to come up with 
ideas how to recover to a consistent state. We have to await their report and 
suggestions before deciding what to do in RDF.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-29 Thread JanZerebecki
JanZerebecki added a comment.

All the software I checked that handles XSD types implement XSD 1.0 BCE years. 
Together we know of nothing that implements XSD 1.1 BCE years. I'd suggest we 
produce output for the RDF tools that exist.

 The calendar model used for saving the data is always the proleptic 
 Gregorian calendar according to ISO 8601.


And that does not refer to the 2000 version, so  is an invalid year. I'm 
not aware of any changes there. It is just not rejected during validation, like 
a Feb 31.

 We have to await their report and suggestions before deciding what to do in 
 RDF.


The only review I'm aware of is related to dates in Julian calendar only.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-29 Thread daniel
daniel added a comment.

I agree that we should gather more data before continuing the discussion about 
the interpretation of dates stored in wikidata.

In https://phabricator.wikimedia.org/T94064#1160748, @mkroetzsch wrote:

  @mkroetzsch Do you know of some widely used software that implements XSD 
  1.1 handling of BCE dates?


 Many applications that process dates are based on ISO rather than on XSD.


Which version of ISO 8901, though?

From Wikipedia: //ISO 8601:2004 (and previously ISO 8601:2000, but not ISO 
8601:1988) explicitly uses astronomical year numbering in its date reference 
systems.// https://en.wikipedia.org/wiki/0_%28year%29#ISO_8601

ISO doesn't save us, since they made the same breaking change, just a few years 
earlier. If the spec or library docs don't explicitly say whether they use 
8901:2000 or later, we might think they use ISO 8601:1988, which does the same 
as XSD 1.0: it does not allow year zero, and counts -1 as 1 BC.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, daniel
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-27 Thread JanZerebecki
JanZerebecki added a comment.

About representation of years BCE: Thank you for starting that mailing list 
thread. It seem that it tends towards saying that the normative thing for 
SPARQL 1.1 is XSD 1.1 with the reasoning that XSD 1.1 retroactively changes 
anything that refers to XSD 1.0 as otherwise SPARQL 1.1 Query Language and 
Entailment Regimes would disagree. If that remains, the question is what to do 
regarding implementation reality which because of Java might lean towards XSD 
1.0. Implementations of XPath functions (like libxslt) and XSD validators (like 
libxml) are probably also relevant.

Virtuoso seems to implement XSD 1.0:

  prefix xsd: http://www.w3.org/2001/XMLSchema#
  
  SELECT ?date
  WHERE {
BIND ( 0001-01-01T00:00:00^^xsd:dateTime - 
-0002-01-01T00:00:00^^xsd:dateTime AS ?date)
  }

  res:value 
datatype=http://www.w3.org/2001/XMLSchema#integer;63158400/res:value

= (365+366)*24*60*60

  Virtuoso 22007 Error DT006: Cannot convert -01-01T00:00:00-03:00 to 
datetime : Incorrect year value
  
  SPARQL query:
  define sql:big-data-const 0 
  #output-format:application/rdf+xml
  prefix xsd: http://www.w3.org/2001/XMLSchema#
  
  SELECT ?date
  WHERE {
BIND ( -0001-01-01T00:00:00^^xsd:dateTime AS ?date)
  }

Maybe we should wait on acting on this until Java XMLGregorianCalendar and 
libxml/xslt were changed?


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, JanZerebecki, Aklapper, 
jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-27 Thread JanZerebecki
JanZerebecki added a comment.

The lexical representation where the year fraction is 0 has undefined meaning 
in Wikibase, so we can not assume it is a month-day without a year. I think the 
easiest way is to represent is as a date with undefined meaning as string, like 
we should additionally do for things we munge to near values with meaning like 
Feb 31. Then go and define a month-day without year in Wikibase.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, JanZerebecki
Cc: Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, JanZerebecki, Aklapper, 
jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-27 Thread Smalyshev
Smalyshev added a comment.

 The lexical representation where the year fraction is 0 has undefined meaning 
 in Wikibase,


True. That's the complication - it can both be used for year before 1AD and 
for we don't know what year it was but it was July 4th.

As we probably will implement custom date parser in BlazeGraph, we can have 
special handling for year 0. The question is *what* that handling should be - 
given the above especially. We don't want a person having born on JUly 4th 
unknown year show up in the list of persons born in 1 BCE.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, JanZerebecki, Aklapper, 
jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-27 Thread mkroetzsch
mkroetzsch added a comment.

   we don't know what year it was but it was July 4th


Ouch. Where has this been designed? Can you point to the specification of this?

@Denny, is this intended? Dates without a year are extremely hard to handle in 
queries and don't work at all like the normal dates we have. This should be a 
different datatype.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev, mkroetzsch
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T94064: Date of +0000-01-01 is allowed but undefined in wikibase but is not allowed in xsd:dateTime as implemented by blazegraph

2015-03-27 Thread Smalyshev
Smalyshev added a comment.

@mkroetzsch I don't think it is specified anywhere, it's just what people do. 
See https://en.wikipedia.org/wiki/Marissa_Stott and its wikidata item, and 
another one above. If you just look for items with date 0 in RDF, I'm sure many 
of them are exactly that. I don't like it but that's what we have in the data, 
so we need to decide what to do with it.

OTOH, while I agree we should eventually be RDF 1.1 compatible, it does not 
mean we're obliged to represent all dates in the DB as xsd:dateTime, and I 
think we are allowed to  tweak xsd:dateTime interpretation somewhat. And we 
don't want to get wrong query results.

For now, until we figured out the whole how dates are represented internally 
thing, I think we should take this road for simple values (deep values always 
have original string):

1. AD dates go to xsd:dateTime, if there's an invalid date like February 31, we 
make it last day of February that year, and so on. That would allow us to 
range-search it.
2. Year 0 is invalid date for now, until we decide what to do with it.
3. Negative years are translated into xsd:dateTime as is, i.e. year -1 in data 
is year -1 in RDF's xsd:dateTime

The last one may be dangerous if the loading tool goes after RDF 1.1 as 1 BCE 
in the data is rendered as -0001. So we can in theory dump it as - but then 
most RDF tools wouldn't load negative dates correctly. 
No idea currently what to do with this.


TASK DETAIL
  https://phabricator.wikimedia.org/T94064

REPLY HANDLER ACTIONS
  Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign 
username.

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Smalyshev
Cc: Lydia_Pintscher, Denny, Manybubbles, daniel, mkroetzsch, Smalyshev, 
JanZerebecki, Aklapper, jkroll, Wikidata-bugs, Jdouglas, aude, GWicke



___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs