[Wikidata] Primary Sources Tool Backend Updates

[Sebastian Schaffert]

I've been reading mostly the archives and the GitHub tickets so far, but
given the interest in the Primary Sources Tool maybe it's time I joined the
mailinglist ;-)

Following up on the discussions the last weeks I added two new features to
the backend:

1. for any request listing statements it is now possible to filter by
state, with the default being "unapproved"

For example, you can select 10 random statements that have been marked
"wrong" (i.e. rejected in the UI) with

curl -i "
https://tools.wmflabs.org/wikidata-primary-sources/statements/any?state=wrong
"

or retrieve all approved statements with

curl -i "
https://tools.wmflabs.org/wikidata-primary-sources/statements/all?state=approved
"

(NB: the /all endpoint is using paging, use the offset= and limit=
parameters to control how much is returned)

the different acceptable states are defined in
https://github.com/google/primarysources/blob/master/backend/Statement.h#L14


2. all statements that already had some form of interaction (e.g. have been
approved or rejected) now contain a new JSON field "activities" listing the
activities acting on the statement; even though usually there will be at
most one activity (i.e. approved or rejected), the system stores (and
already stored since we launched it) a complete history, e.g. for
transitions like unapproved -> wrong -> unapproved -> approved.

You can try it out by retrieving a random selection of statements in other
states than "unapproved", e.g. as before:

curl -i "
https://tools.wmflabs.org/wikidata-primary-sources/statements/any?state=wrong
"

will give you results like:

{
"activities" : [
{
"state" : "wrong",
"timestamp" : "+2015-05-09T14:26:45Z/14",
"user" : "Hoo man"
}
]
,
"dataset" : "freebase",
"format" : "v1",
"id" : 31,
"state" : "wrong",
"statement" : "Q1702409\tP27\tQ145\tS854\t\"
http://www.astrotheme.com/astrology/Warren_Mitchell\"";,
"upload" : 0
}

Hope it is useful ;-)

Otherwise let me know if you are interested in other analysis data. I'll
try adding features as time permits.

Cheers!



-- 
Dr. Sebastian Schaffert | GMail Site Reliability Manager |
schaff...@google.com | +41 44 668 06 25
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata

Re: [Wikidata] Primary sources tool "reject claim" broken?

[Sebastian Schaffert]

Hi Thomas and all,

there might be a caching issue here. That part of the code is here:
https://github.com/google/primarysources/blob/master/backend/service/SourcesToolBackend.cc#L115
and
it still seems right to me, but I'll check again.

I won't have much time in the next days though :( I'll give it one hour no,
maybe I discover something.

Cheers,

Sebastian

On Tue, May 31, 2016 at 9:17 AM Thomas Steiner  wrote:

> Hi Markus and Marco, all,
>
> Thanks for your support of and caring for the Primary Sources Tool.
> Please find my replies inline.
>
> > Dear ,
>
> I guess the core team still does, with the caveat explained by Denny
> in [1], the tl;dr is that we work on it on top of our regular jobs and
> that we are happy to hand it over to folks with more time on their
> hands.
>
> > The PS tool seems to break more and more. Besides the persisting issue
> with
> > duplicated claims being offered (even if they are already stored), there
> is
> > now also the issue that claims cannot be rejected. If I reject a claim,
> the
> > page reloads, but the suggestion still shows up after that.
>
> I checked both problems. It seems the writes from the front-end
> somehow do not make it to the back-end. I opened a random item Q632229
> and approved and rejected claims. The approval went through just fine
> [2].
>
> (i) However, I could reproduce the duplicate claims being shown, the
> reason is that the uniqueness comparison does not take references into
> account [3], a known @ToDo up for grabs.
> (ii) I could also in some cases reproduce the non-rejectable claims
> issue. I repeated disapproved statement 868483 [4], but if you query
> the back-end for incoming Freebase statements for Q632229, it keeps
> coming back as "unapproved" [5] (search for "868483").
>
> For (i), if someone wants to tackle this, happy to merge their Pull
> Request. For (ii), Sebastian, do you have a suspicion why this could
> be the case?
>
> Thanks,
> Tom
>
> --
> [1]
> https://lists.wikimedia.org/pipermail/wikidata/2016-February/008316.html
> [2]
> https://www.wikidata.org/w/index.php?title=Q632229&type=revision&diff=341928371&oldid=316931253
> [3]
> https://github.com/google/primarysources/blob/master/frontend/freebase2wikidata.js#L805
> [4] https://tools.wmflabs.org/wikidata-primary-sources/statements/868483
> [5] https://tools.wmflabs.org/wikidata-primary-sources/entities/Q632229
>
> --
> Dr. Thomas Steiner, Employee (http://blog.tomayac.com,
> https://twitter.com/tomayac)
>
> Google Germany GmbH, ABC-Str. 19, 20354 Hamburg, Germany
> Managing Directors: Matthew Scott Sucherman, Paul Terence Manicle
> Registration office and registration number: Hamburg, HRB 86891
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.29 (GNU/Linux)
>
>
> iFy0uwAntT0bE3xtRa5AfeCheCkthAtTh3reSabiGbl0ck0fjumBl3DCharaCTersAttH3b0ttom
> hTtPs://xKcd.cOm/1181/
> -END PGP SIGNATURE-
>
___
Wikidata mailing list
Wikidata@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata