Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[James Heald]

Hi Léa,

Thanks to all the team for this.

I've proposed a property,

https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_for_class

To make this work, is it possible to have a Shape Expression as the 
value of a statement on Wikidata (and the RDF dump, and WDQS) ?


Is there a timescale in which this should become possible ?

Thanks,

   James.




On 28/05/2019 17:04, Léa Lacroix wrote:

Hello all,

As previously announced, we just released shape expressions on Wikidata.
You can for example have a look at E10, the shape for human
, or create a new
EntitySchema .

A few useful links:

- WikiProject ShEx

- introduction to ShEx 
- more details about the language 
- More information about how to create a Schema


- Phabricator tag: shape-expressions

- User script

to highlight items and properties in the schema code and turn the IDs into
links

If you have any question or encounter issues, feel free to ping me. Cheers,

Léa

On Sun, 19 May 2019 at 15:32, Léa Lacroix  wrote:


Hello all,

After several months of development and testing together with the WikiProject
ShEx , Shape
Expressions are about to be enabled on Wikidata.
*First of all, what are Shape Expressions?*

ShEx (Q29377880)  is a concise,
formal modeling and validation language for RDF structures. Shape
Expressions can be used to define shapes within the RDF graph. In the case
of Wikidata, this would be sets of properties, qualifiers and references
that describe the domain being modeled.

See also:

- a short video about ShEx
 made by community
members during the Wikimedia hackathon 2019
- introduction to ShEx 
- more details about the language 

*What can it be used for?*

On Wikidata, the main goal of Shape Expressions would be to describe what
the basic structure of an item would be. For example, for a human, we
probably want to have a date of birth, a place of birth, and many other
important statements. But we would also like to make sure that if a
statement with the property “children” exists, the value(s) of this
property should be humans as well. Schemas will describe in detail what is
expected in the structure of items, statements and values of these
statements.

Once Schemas are created for various types of items, it is possible to
test some existing items against the Schema, and highlight possible errors
or lack of information. Subsets of the Wikidata graph can be tested to see
whether or not they conform to a specific shape through the use of
validation tools. Therefore, Schemas will be very useful to help the
editors improving the data quality. We imagine this to be especially useful
for wiki projects to more easily discuss and ensure the modeling of items
in their domain. In the spirit of Wikidata not restricting the world, Shape
Expressions are a tool to highlight, not prevent, errors.

On top of this, one could imagine other uses of Schemas in the future, for
example building a tool that would suggest, when creating a new item, what
would be the basic structure for this item, and helping adding statements
or values. A bit like this existing tool, Cradle
, that is currently
not based on ShEx.
*What is going to change on Wikidata?*

- A new extension will be added to Wikidata: EntitySchema
, defining the
Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can fill
with a syntax called ShEx Compact Syntax (ShExC)
. You can see an example here
.
- The external tool shex-simple


is directly linked from the Schema pages in order to check entities of your
choice against the schema.

*When is this happening?*

Schemas wil

Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[Antoine Zimmermann]

Hello,


Could you explain why the non-standard ShEx has been chosen rather than 
the W3C Recommendation SHACL?


I would assume that if one has several options for bringing a 
functionality to something that largely promotes interoperability (like 
Wikidata), the default choice should be a standard, and /only if/ one 
has a carefully crafted argumentation to reject it, one would opt for 
something else.


For those who may not know, the W3C RDF Data Shapes Working Group worked 
between 2014 and 2017 on defining a standard for describing data shapes 
in RDF. ShEx existed already and was a candidate for standardisation. 
Eventually, another standard emerged, Shapes Constraint Language (SHACL, 
see https://www.w3.org/TR/shacl/).


Disclaimer: I did not contribute to either SHACL or ShEx, and I do not 
know them enough to judge which one is better.



Best,
--AZ


On 19/05/2019 15:32, Léa Lacroix wrote:

Hello all,

After several months of development and testing together with the 
WikiProject ShEx 
, Shape 
Expressions are about to be enabled on Wikidata.


*First of all, what are Shape Expressions?*

ShEx (Q29377880)  is a concise, 
formal modeling and validation language for RDF structures. Shape 
Expressions can be used to define shapes within the RDF graph. In the 
case of Wikidata, this would be sets of properties, qualifiers and 
references that describe the domain being modeled.


See also:

  * a short video about ShEx
 made by community
members during the Wikimedia hackathon 2019
  * introduction to ShEx 
  * more details about the language 

*What can it be used for?*

On Wikidata, the main goal of Shape Expressions would be to describe 
what the basic structure of an item would be. For example, for a human, 
we probably want to have a date of birth, a place of birth, and many 
other important statements. But we would also like to make sure that if 
a statement with the property “children” exists, the value(s) of this 
property should be humans as well. Schemas will describe in detail what 
is expected in the structure of items, statements and values of these 
statements.


Once Schemas are created for various types of items, it is possible to 
test some existing items against the Schema, and highlight possible 
errors or lack of information. Subsets of the Wikidata graph can be 
tested to see whether or not they conform to a specific shape through 
the use of validation tools. Therefore, Schemas will be very useful to 
help the editors improving the data quality. We imagine this to be 
especially useful for wiki projects to more easily discuss and ensure 
the modeling of items in their domain. In the spirit of Wikidata not 
restricting the world, Shape Expressions are a tool to highlight, not 
prevent, errors.


On top of this, one could imagine other uses of Schemas in the future, 
for example building a tool that would suggest, when creating a new 
item, what would be the basic structure for this item, and helping 
adding statements or values. A bit like this existing tool, Cradle 
, that is currently 
not based on ShEx.


*What is going to change on Wikidata?*

  * A new extension will be added to Wikidata: EntitySchema
, defining
the Schema namespace and its behavior as well as special pages
related to it.
  * A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
  * The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can
fill with a syntax called ShEx Compact Syntax (ShExC)
. You can see an example here
.
  * The external tool shex-simple


is directly linked from the Schema pages in order to check entities
of your choice against the schema.

*When is this happening?*

Schemas will be enabled on on test.wikidata.org 
 on May 21st and on wikidata.org 
 on May 28th. After this release, they will be 
integrated to the regular maintenance just like the rest of Wikidata’s 
features.


*How can you help?*

  * Before the release, you can try to edit or create Shape Expressions
on our test system 
  * If you find any issue or feature you’d like to have, feel free to
create a new task on Phabricator with the tag |shape-expre

Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[Magnus Sälgö]

Very interesting as Wikidata starts beeing part of more external data “flows” I 
would also like to see that we easy can tell
* this is the schema we use inside Wikidata
* this is external related schemas other organisations has created for this 
type of data
* how this WD schema relate to an external schema. What parts we map etc.

In Sweden the goverment speaks about basic data (swe. Grunddata) that they will 
define that I hope the data in Wikidata can “plug-in” and add value to.

Regards
Magnus Sälgö
0046-705937579
salg...@msn.com

29 maj 2019 kl. 09:44 skrev Léa Lacroix 
mailto:lea.lacr...@wikimedia.de>>:

Thanks for your feedback! There is already a ticket about adding a new data 
type allowing to link EntitySchemas from statements: 
https://phabricator.wikimedia.org/T214884
If we don't encounter any major technical issue, this could be done in the 
incoming weeks.

On Tue, 28 May 2019 at 19:18, James Heald 
mailto:jpm.he...@gmail.com>> wrote:
Hi Léa,

Thanks to all the team for this.

I've proposed a property,

https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_for_class

To make this work, is it possible to have a Shape Expression as the
value of a statement on Wikidata (and the RDF dump, and WDQS) ?

Is there a timescale in which this should become possible ?

Thanks,

James.




On 28/05/2019 17:04, Léa Lacroix wrote:
> Hello all,
>
> As previously announced, we just released shape expressions on Wikidata.
> You can for example have a look at E10, the shape for human
> , or create a new
> EntitySchema .
>
> A few useful links:
>
> - WikiProject ShEx
> 
> - introduction to ShEx 
> - more details about the language 
> - More information about how to create a Schema
> 
> 
> - Phabricator tag: shape-expressions
> 
> - User script
> 
> to highlight items and properties in the schema code and turn the IDs into
> links
>
> If you have any question or encounter issues, feel free to ping me. Cheers,
>
> Léa
>
> On Sun, 19 May 2019 at 15:32, Léa Lacroix 
> mailto:lea.lacr...@wikimedia.de>> wrote:
>
>> Hello all,
>>
>> After several months of development and testing together with the WikiProject
>> ShEx , Shape
>> Expressions are about to be enabled on Wikidata.
>> *First of all, what are Shape Expressions?*
>>
>> ShEx (Q29377880)  is a concise,
>> formal modeling and validation language for RDF structures. Shape
>> Expressions can be used to define shapes within the RDF graph. In the case
>> of Wikidata, this would be sets of properties, qualifiers and references
>> that describe the domain being modeled.
>>
>> See also:
>>
>> - a short video about ShEx
>>  made by community
>> members during the Wikimedia hackathon 2019
>> - introduction to ShEx 
>> - more details about the language 
>>
>> *What can it be used for?*
>>
>> On Wikidata, the main goal of Shape Expressions would be to describe what
>> the basic structure of an item would be. For example, for a human, we
>> probably want to have a date of birth, a place of birth, and many other
>> important statements. But we would also like to make sure that if a
>> statement with the property “children” exists, the value(s) of this
>> property should be humans as well. Schemas will describe in detail what is
>> expected in the structure of items, statements and values of these
>> statements.
>>
>> Once Schemas are created for various types of items, it is possible to
>> test some existing items against the Schema, and highlight possible errors
>> or lack of information. Subsets of the Wikidata graph can be tested to see
>> whether or not they conform to a specific shape through the use of
>> validation tools. Therefore, Schemas will be very useful to help the
>> editors improving the data quality. We imagine this to be especially useful
>> for wiki projects to more easily discuss and ensure the modeling of items
>> in their domain. In the spirit of Wikidata not restricting the world, Shape
>> Expressions are a tool to highlight, not prevent, errors.
>>
>> On top of this, one could imagine other uses of Schemas in the future, for
>> example building a tool that would suggest, when creating a new item, what
>> would be the basic structure for this item, and helping adding statements
>> or values. A bit like this existing t

Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[Léa Lacroix]

Thanks for your feedback! There is already a ticket about adding a new data
type allowing to link EntitySchemas from statements:
https://phabricator.wikimedia.org/T214884
If we don't encounter any major technical issue, this could be done in the
incoming weeks.

On Tue, 28 May 2019 at 19:18, James Heald  wrote:

> Hi Léa,
>
> Thanks to all the team for this.
>
> I've proposed a property,
>
>
> https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_for_class
>
> To make this work, is it possible to have a Shape Expression as the
> value of a statement on Wikidata (and the RDF dump, and WDQS) ?
>
> Is there a timescale in which this should become possible ?
>
> Thanks,
>
> James.
>
>
>
>
> On 28/05/2019 17:04, Léa Lacroix wrote:
> > Hello all,
> >
> > As previously announced, we just released shape expressions on Wikidata.
> > You can for example have a look at E10, the shape for human
> > , or create a new
> > EntitySchema .
> >
> > A few useful links:
> >
> > - WikiProject ShEx
> > 
> > - introduction to ShEx 
> > - more details about the language 
> > - More information about how to create a Schema
> > <
> https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
> >
> > - Phabricator tag: shape-expressions
> > 
> > - User script
> > <
> https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js>
> > to highlight items and properties in the schema code and turn the
> IDs into
> > links
> >
> > If you have any question or encounter issues, feel free to ping me.
> Cheers,
> >
> > Léa
> >
> > On Sun, 19 May 2019 at 15:32, Léa Lacroix 
> wrote:
> >
> >> Hello all,
> >>
> >> After several months of development and testing together with the
> WikiProject
> >> ShEx , Shape
> >> Expressions are about to be enabled on Wikidata.
> >> *First of all, what are Shape Expressions?*
> >>
> >> ShEx (Q29377880)  is a
> concise,
> >> formal modeling and validation language for RDF structures. Shape
> >> Expressions can be used to define shapes within the RDF graph. In the
> case
> >> of Wikidata, this would be sets of properties, qualifiers and references
> >> that describe the domain being modeled.
> >>
> >> See also:
> >>
> >> - a short video about ShEx
> >>  made by community
> >> members during the Wikimedia hackathon 2019
> >> - introduction to ShEx 
> >> - more details about the language 
> >>
> >> *What can it be used for?*
> >>
> >> On Wikidata, the main goal of Shape Expressions would be to describe
> what
> >> the basic structure of an item would be. For example, for a human, we
> >> probably want to have a date of birth, a place of birth, and many other
> >> important statements. But we would also like to make sure that if a
> >> statement with the property “children” exists, the value(s) of this
> >> property should be humans as well. Schemas will describe in detail what
> is
> >> expected in the structure of items, statements and values of these
> >> statements.
> >>
> >> Once Schemas are created for various types of items, it is possible to
> >> test some existing items against the Schema, and highlight possible
> errors
> >> or lack of information. Subsets of the Wikidata graph can be tested to
> see
> >> whether or not they conform to a specific shape through the use of
> >> validation tools. Therefore, Schemas will be very useful to help the
> >> editors improving the data quality. We imagine this to be especially
> useful
> >> for wiki projects to more easily discuss and ensure the modeling of
> items
> >> in their domain. In the spirit of Wikidata not restricting the world,
> Shape
> >> Expressions are a tool to highlight, not prevent, errors.
> >>
> >> On top of this, one could imagine other uses of Schemas in the future,
> for
> >> example building a tool that would suggest, when creating a new item,
> what
> >> would be the basic structure for this item, and helping adding
> statements
> >> or values. A bit like this existing tool, Cradle
> >> , that is currently
> >> not based on ShEx.
> >> *What is going to change on Wikidata?*
> >>
> >> - A new extension will be added to Wikidata: EntitySchema
> >> , defining
> the
> >> Schema namespace and its behavior as well as special pages related
> to it.
> >> - A new entity type, EntitySchema, will be enabled to store Shape
> >> Expressions. 

Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[James Hare]

In other words: the benefits of standardized schemata but with less
gatekeeping and exclusivity. Cheers!


On Sun, May 19, 2019 at 3:32 PM Léa Lacroix 
wrote:

> Hello all,
>
> After several months of development and testing together with the WikiProject
> ShEx , Shape
> Expressions are about to be enabled on Wikidata.
> *First of all, what are Shape Expressions?*
>
> ShEx (Q29377880)  is a concise,
> formal modeling and validation language for RDF structures. Shape
> Expressions can be used to define shapes within the RDF graph. In the case
> of Wikidata, this would be sets of properties, qualifiers and references
> that describe the domain being modeled.
>
> See also:
>
>- a short video about ShEx
> made by community
>members during the Wikimedia hackathon 2019
>- introduction to ShEx 
>- more details about the language 
>
> *What can it be used for?*
>
> On Wikidata, the main goal of Shape Expressions would be to describe what
> the basic structure of an item would be. For example, for a human, we
> probably want to have a date of birth, a place of birth, and many other
> important statements. But we would also like to make sure that if a
> statement with the property “children” exists, the value(s) of this
> property should be humans as well. Schemas will describe in detail what is
> expected in the structure of items, statements and values of these
> statements.
>
> Once Schemas are created for various types of items, it is possible to
> test some existing items against the Schema, and highlight possible errors
> or lack of information. Subsets of the Wikidata graph can be tested to see
> whether or not they conform to a specific shape through the use of
> validation tools. Therefore, Schemas will be very useful to help the
> editors improving the data quality. We imagine this to be especially useful
> for wiki projects to more easily discuss and ensure the modeling of items
> in their domain. In the spirit of Wikidata not restricting the world, Shape
> Expressions are a tool to highlight, not prevent, errors.
>
> On top of this, one could imagine other uses of Schemas in the future, for
> example building a tool that would suggest, when creating a new item, what
> would be the basic structure for this item, and helping adding statements
> or values. A bit like this existing tool, Cradle
> , that is currently
> not based on ShEx.
> *What is going to change on Wikidata?*
>
>- A new extension will be added to Wikidata: EntitySchema
>, defining the
>Schema namespace and its behavior as well as special pages related to it.
>- A new entity type, EntitySchema, will be enabled to store Shape
>Expressions. Schemas will be identified with the letter E.
>- The Schemas will have multilingual labels, descriptions and aliases
>(quite similar to the termbox on Items), and the schema text one can fill
>with a syntax called ShEx Compact Syntax (ShExC)
>. You can see an example here
>.
>- The external tool shex-simple
>
> 
>is directly linked from the Schema pages in order to check entities of your
>choice against the schema.
>
> *When is this happening?*
>
> Schemas will be enabled on on test.wikidata.org on May 21st and on
> wikidata.org on May 28th. After this release, they will be integrated to
> the regular maintenance just like the rest of Wikidata’s features.
> *How can you help?*
>
>- Before the release, you can try to edit or create Shape Expressions
>on our test system 
>- If you find any issue or feature you’d like to have, feel free to
>create a new task on Phabricator with the tag shape-expressions
>- Once Schemas are enabled, you can discuss about it on your favorite
>wikiprojects: for example, what types of items would you like to model?
>- You can also get more information about how to create a Schema
>
> 
>
> *See also: *
>
>- Main Phabricator board
>
>- Technical documentation of the extension
>
>- To enhance the interface, you can use this user script
>
>to highlight items and propert

Re: [Wikidata-tech] [Wikidata] Shape Expressions arrive on Wikidata on May 28th

[Luca Martinelli]

This is fantastic news, as always. :) I've got a question: can/will those
entities also be recalled from Lua templates to help build infoboxes?
(Sorry for being cryptic, I'm busy and cannot further explain at the
moment, but I cannot help my imagination run wild!) :)))

L.

Il dom 19 mag 2019, 15:32 Léa Lacroix  ha scritto:

> Hello all,
>
> After several months of development and testing together with the WikiProject
> ShEx , Shape
> Expressions are about to be enabled on Wikidata.
> *First of all, what are Shape Expressions?*
>
> ShEx (Q29377880)  is a concise,
> formal modeling and validation language for RDF structures. Shape
> Expressions can be used to define shapes within the RDF graph. In the case
> of Wikidata, this would be sets of properties, qualifiers and references
> that describe the domain being modeled.
>
> See also:
>
>- a short video about ShEx
> made by community
>members during the Wikimedia hackathon 2019
>- introduction to ShEx 
>- more details about the language 
>
> *What can it be used for?*
>
> On Wikidata, the main goal of Shape Expressions would be to describe what
> the basic structure of an item would be. For example, for a human, we
> probably want to have a date of birth, a place of birth, and many other
> important statements. But we would also like to make sure that if a
> statement with the property “children” exists, the value(s) of this
> property should be humans as well. Schemas will describe in detail what is
> expected in the structure of items, statements and values of these
> statements.
>
> Once Schemas are created for various types of items, it is possible to
> test some existing items against the Schema, and highlight possible errors
> or lack of information. Subsets of the Wikidata graph can be tested to see
> whether or not they conform to a specific shape through the use of
> validation tools. Therefore, Schemas will be very useful to help the
> editors improving the data quality. We imagine this to be especially useful
> for wiki projects to more easily discuss and ensure the modeling of items
> in their domain. In the spirit of Wikidata not restricting the world, Shape
> Expressions are a tool to highlight, not prevent, errors.
>
> On top of this, one could imagine other uses of Schemas in the future, for
> example building a tool that would suggest, when creating a new item, what
> would be the basic structure for this item, and helping adding statements
> or values. A bit like this existing tool, Cradle
> , that is currently
> not based on ShEx.
> *What is going to change on Wikidata?*
>
>- A new extension will be added to Wikidata: EntitySchema
>, defining the
>Schema namespace and its behavior as well as special pages related to it.
>- A new entity type, EntitySchema, will be enabled to store Shape
>Expressions. Schemas will be identified with the letter E.
>- The Schemas will have multilingual labels, descriptions and aliases
>(quite similar to the termbox on Items), and the schema text one can fill
>with a syntax called ShEx Compact Syntax (ShExC)
>. You can see an example here
>.
>- The external tool shex-simple
>
> 
>is directly linked from the Schema pages in order to check entities of your
>choice against the schema.
>
> *When is this happening?*
>
> Schemas will be enabled on on test.wikidata.org on May 21st and on
> wikidata.org on May 28th. After this release, they will be integrated to
> the regular maintenance just like the rest of Wikidata’s features.
> *How can you help?*
>
>- Before the release, you can try to edit or create Shape Expressions
>on our test system 
>- If you find any issue or feature you’d like to have, feel free to
>create a new task on Phabricator with the tag shape-expressions
>- Once Schemas are enabled, you can discuss about it on your favorite
>wikiprojects: for example, what types of items would you like to model?
>- You can also get more information about how to create a Schema
>
> 
>
> *See also: *
>
>- Main Phabricator board
>
>- Technical documentation of the extension
>
>-