Re: [basex-talk] from xquery:parse to data paths ...

[Christian Grün]

Hi Hans-Jürgen,

One reason why I would favor to have an XQuery implementation first is
because it would allow us to define the semantics behind this
function. There are currently lots of questions that are unclear to
me, so it’s mostly conceptual questions that would need to be solved
before such a function could be realized.

I had a closer look at your examples:

let $query := "/descendant::OutboundFlight[Status =
'Booked'][1]/FlightDeparture/DepartureTime"

let $query := 
"/descendant::OutboundFlight[FlightDeparture/DepartureDate][1]/FlightDeparture/DepartureDate"
return xquery:data-paths($query)
=>
//OutboundFlight/FlightDeparture/DepartureTime,
//OutboundFlight/Status


Probably the result of the second query refers to your first example, right?

To be strict, "/descendant::OutboundFlight" is not identical to
"//OutboundFlight"; but it is equivalent. So I would have expected to
get the following paths as output:

/descendant::OutboundFlight/child::FlightDeparture/child::DepartureTime,
/descendant::OutboundFlight/child::Status

"//OF[1]" could lead to:

/descendant-or-self::node()/child::OF

Next, we would need to sort out if/how to handle axes other than
descendant-or-self, descendant and child. Moreover, as XQuery allows
us to have all kinds of expression nearly everywhere, sth. like "A[B =
'c']" or "A[B]/C" are rather special (albeit common) query patterns
that could be written in many different ways. The most obvious
patterns could probably nailed down and realized pretty quickly, but
there would be a lot of potenzial for optimizing the query output and
considering corner cases, and this is usually something that costs a
lot of time.

As I assume that you have some specific patterns in mind that would be
helpful for you, would you be interested in providing an initial
solution in XQuery that we could adopt as Java function later on?

Christian

Re: [basex-talk] from xquery:parse to data paths ...

[Christian Grün]

Hi Hans-Jürgen,

Sounds like an interesting idea. As the output of xquery:parse is XML,
the xquery:data-paths function could probably be written in XQuery
itself?

Best,
Christian
_

On Sat, Feb 20, 2016 at 12:24 AM, Hans-Juergen Rennau  wrote:
> Dear BaseX team,
>
> perhaps I hear an opinion about the following idea.
>
> (1) The function xquery:parse is already immensely useful as it allows to
> *validate* XPath expressions, as for example used in configuration data. (A
> practical example: the XPath expressions used in JMeter [1] test plans in
> order to extract message data and define test assertions.)
>
> (2) Given the XSDs of a Web Service (or other application), it is possible
> to determine all valid data paths (e.g. /a/b/c is valid, /a/b/C is not
> valid).
>
> (3) If there were a reliable way to map the output of xquery:parse to the
> implied data paths, then one could use xquery:parse to validate xpath
> expression not only for syntactic correctness (which is already very much!),
> but also for consistency with application XSDs. This would be extremly
> usefuly as XSDs evolve and a checking if configuration-based XPath
> expressions must be adapted is otherwise very difficult to achieve in any
> systematic way.
>
> (4) Looking at the xquery:parse output, it seems certainly feasible to write
> such a mapping (xquery:parse output => data paths); but the problem I see is
> that the format is not guaranteed to be stable, as it is no standard. (And
> it is probably not described.) If BaseX provided such a functionality as an
> additional extension function (e.g. xquery:data-paths($query as xs:string))
> you would simply be - heros.
>
> Cheers,
> Hans-Jürgen
>
> [1] http://jmeter.apache.org/
>
> PS: Illustrative example.
>
> let $query := "/descendant::OutboundFlight[Status =
> 'Booked'][1]/FlightDeparture/DepartureTime"
> return xquery:parse($query)
> =>
> 
>   
> 
>   
>   
> 
>   
> 
>   
>   
> 
> 
>   
>   
>   
> 
>   
> 
>
> let $query :=
> "/descendant::OutboundFlight[FlightDeparture/DepartureDate][1]/FlightDeparture/DepartureDate"
> return xquery:data-paths($query)
> =>
> //OutboundFlight/FlightDeparture/DepartureTime,
> //OutboundFlight/Status
>

[basex-talk] from xquery:parse to data paths ...

[Hans-Juergen Rennau]

Dear BaseX team,
perhaps I hear an opinion about the following idea.
(1) The function xquery:parse is already immensely useful as it allows to 
*validate* XPath expressions, as for example used in configuration data. (A 
practical example: the XPath expressions used in JMeter [1] test plans in order 
to extract message data and define test assertions.)
(2) Given the XSDs of a Web Service (or other application), it is possible to 
determine all valid data paths (e.g. /a/b/c is valid, /a/b/C is not valid).
(3) If there were a reliable way to map the output of xquery:parse to the 
implied data paths, then one could use xquery:parse to validate xpath 
expression not only for syntactic correctness (which is already very much!), 
but also for consistency with application XSDs. This would be extremly usefuly 
as XSDs evolve and a checking if configuration-based XPath expressions must be 
adapted is otherwise very difficult to achieve in any systematic way.
(4) Looking at the xquery:parse output, it seems certainly feasible to write 
such a mapping (xquery:parse output => data paths); but the problem I see is 
that the format is not guaranteed to be stable, as it is no standard. (And it 
is probably not described.) If BaseX provided such a functionality as an 
additional extension function (e.g. xquery:data-paths($query as xs:string)) you 
would simply be - heros.
Cheers,Hans-Jürgen

[1] http://jmeter.apache.org/
PS: Illustrative example.
let $query := "/descendant::OutboundFlight[Status = 
'Booked'][1]/FlightDeparture/DepartureTime"return 
xquery:parse($query)=>
  
    
  
  
    
  
    
  
  
    
    
  
  
  
    
  


let $query := 
"/descendant::OutboundFlight[FlightDeparture/DepartureDate][1]/FlightDeparture/DepartureDate"return
 
xquery:data-paths($query)=>//OutboundFlight/FlightDeparture/DepartureTime,//OutboundFlight/Status

Re: [basex-talk] Automated Docker Build: basex/basexhttp

[Jens Erat]

> The problem was that I wanted to add files to the /webapp directory, but
> once the Dockerfile defines it as volume you can't modify it as part of
> the container definition, you can only mount to it from outside.
> 
> My general approach is to have two images, one without the volumes, for
> use in defining derived containers, and a working image that defines the
> volumes so they are ready for mounting from other containers or from the
> host.
> 
> I suppose another way to handle it is to not define the volumes at all and
> then do it in Docker-compose scripts (I'm using docker-compose to manage
> all my related containers and provide a convenient way to  manage
> everything through one command).

The way it meant to be is usually adding logic/program code to derived
images, and putting data/information into volumes. How those are managed
depends on personal needs, preferences and the available infrastructure.

THis way you can "throw away" containers at any time, they don't have
any state left any more. To publish a new version, you simply start a
new container from the updated image accessing the data, but don't have
to pass any application logic into the container.

Relying on docker-compose is a good thing to do, it also easons up
migrating to a Docker swarm or other infrastructure management tools.

-- 
Jens Erat
Universität Konstanz
Kommunikations-, Infomations-, Medienzentrum (KIM)
Abteilung Basisdienste
D-78457 Konstanz
Mail: jens.e...@uni-konstanz.de



smime.p7s
Description: S/MIME Cryptographic Signature

Re: [basex-talk] Automated Docker Build: basex/basexhttp

[Eliot Kimber]

The problem was that I wanted to add files to the /webapp directory, but
once the Dockerfile defines it as volume you can't modify it as part of
the container definition, you can only mount to it from outside.

My general approach is to have two images, one without the volumes, for
use in defining derived containers, and a working image that defines the
volumes so they are ready for mounting from other containers or from the
host.

I suppose another way to handle it is to not define the volumes at all and
then do it in Docker-compose scripts (I'm using docker-compose to manage
all my related containers and provide a convenient way to  manage
everything through one command).

Cheers,

E.

Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 2/19/16, 4:27 PM, "Jens Erat"  wrote:

>Hi Eliot,
>
>> Looks like this version addresses the issue I had
>> (creating volumes in the base container made it impossible to add repos
>>or
>> webapps in using Dockerfiles).
>
>that's also one of the issues the earlier Dockerfile of Dirk and Michael
>had -- and I also had issues with that when I started with Docker. It's
>somewhat counter-intuitive that you have to `EXPOSE` ports to publish
>them, but not define `VOLUME`s to bind them.
>
>Defining a volume in a Dockerfile results in a bind-mount of a new,
>empty folder (if not defined otherwise through `--volume[-from]`) during
>container instanciation. So you can very well add files into the image
>-- but they're hidden by that bind mount in the running container.
>
>Regards,
>Jens
>
>
>-- 
>Jens Erat
>
> [phone]: tel:+49-151-56961126
>  [mail]: mailto:em...@jenserat.de
>[jabber]: xmpp:jab...@jenserat.de
>   [web]: http://www.jenserat.de
>
> OpenPGP: 0D69 E11F 12BD BA07 7B37  26AB 4E1F 799A A4FF 2279
>

Re: [basex-talk] Automated Docker Build: basex/basexhttp

[Jens Erat]

Hi Eliot,

> Looks like this version addresses the issue I had
> (creating volumes in the base container made it impossible to add repos or
> webapps in using Dockerfiles).

that's also one of the issues the earlier Dockerfile of Dirk and Michael
had -- and I also had issues with that when I started with Docker. It's
somewhat counter-intuitive that you have to `EXPOSE` ports to publish
them, but not define `VOLUME`s to bind them.

Defining a volume in a Dockerfile results in a bind-mount of a new,
empty folder (if not defined otherwise through `--volume[-from]`) during
container instanciation. So you can very well add files into the image
-- but they're hidden by that bind mount in the running container.

Regards,
Jens


-- 
Jens Erat

 [phone]: tel:+49-151-56961126
  [mail]: mailto:em...@jenserat.de
[jabber]: xmpp:jab...@jenserat.de
   [web]: http://www.jenserat.de

 OpenPGP: 0D69 E11F 12BD BA07 7B37  26AB 4E1F 799A A4FF 2279



smime.p7s
Description: S/MIME Cryptographic Signature

Re: [basex-talk] Automated Docker Build: basex/basexhttp

[Eliot Kimber]

Cool--I'll try it as soon as I can. I have my own BaseX-based container
with a custom Web app working now, but based on my own small mod of an
earlier Dockerfile. Looks like this version addresses the issue I had
(creating volumes in the base container made it impossible to add repos or
webapps in using Dockerfiles).

Cheers,

Eliot

Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 2/19/16, 3:54 PM, "Jens Erat"
 wrote:

>Dear BaseX community,
>
>over the last weeks, interest in Docker utilization with BaseX hevily
>increased, and several images have been proposed. I've already made some
>experience running a BaseX pet project in BaseX for about a year now,
>and together with the BaseX core team created an image based on those
>experiences and current best practices.
>
>
>basex/basexhttp Docker Image
>
>
>Finally there's the official basex/basexhttp Docker image readily
>available on the Docker Hub. It's an automated build directly from
>source, with nightly builds and tagged builds for future BaseX releases
>(starting with 8.4.1/8.5, whatever will come first). It is derived from
>the Maven base image (and thus Debian), and is automatically updated
>when the base images experience updates (like Java security fixes).
>
>- https://hub.docker.com/r/basex/basexhttp/
>- https://github.com/BaseXdb/basex/blob/master/Dockerfile
>
>While the image is named basexhttp and always includes the HTTP server,
>it can also be used for running the plain basexserver or even basexclient.
>
>
>DBA Application Container
>-
>
>As an example for deriving your own application images and also for
>interfacing BaseX for administrative tasks and ad-hoc queries, the DBA
>is also made available as a container.
>
>- https://hub.docker.com/r/basex/dba/
>- 
>https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/dba
>
>
>Documentation
>-
>
>As you're used to, documentation is available in the BaseX wiki.
>
>- http://docs.basex.org/wiki/Docker
>
>- - -
>
>If you've got any feedback, questions or proposals, feel free to get in
>touch with me or the core BaseX team on the usual ways.
>
>Kind regards from Lake Constance, Germany,
>Jens
>
>-- 
>Jens Erat
>
> [phone]: tel:+49-151-56961126
>  [mail]: mailto:em...@jenserat.de
>[jabber]: xmpp:jab...@jenserat.de
>   [web]: http://www.jenserat.de
>
> OpenPGP: 0D69 E11F 12BD BA07 7B37  26AB 4E1F 799A A4FF 2279
>

[basex-talk] Automated Docker Build: basex/basexhttp

[Jens Erat]

Dear BaseX community,

over the last weeks, interest in Docker utilization with BaseX hevily
increased, and several images have been proposed. I've already made some
experience running a BaseX pet project in BaseX for about a year now,
and together with the BaseX core team created an image based on those
experiences and current best practices.


basex/basexhttp Docker Image


Finally there's the official basex/basexhttp Docker image readily
available on the Docker Hub. It's an automated build directly from
source, with nightly builds and tagged builds for future BaseX releases
(starting with 8.4.1/8.5, whatever will come first). It is derived from
the Maven base image (and thus Debian), and is automatically updated
when the base images experience updates (like Java security fixes).

- https://hub.docker.com/r/basex/basexhttp/
- https://github.com/BaseXdb/basex/blob/master/Dockerfile

While the image is named basexhttp and always includes the HTTP server,
it can also be used for running the plain basexserver or even basexclient.


DBA Application Container
-

As an example for deriving your own application images and also for
interfacing BaseX for administrative tasks and ad-hoc queries, the DBA
is also made available as a container.

- https://hub.docker.com/r/basex/dba/
- https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/webapp/dba


Documentation
-

As you're used to, documentation is available in the BaseX wiki.

- http://docs.basex.org/wiki/Docker

- - -

If you've got any feedback, questions or proposals, feel free to get in
touch with me or the core BaseX team on the usual ways.

Kind regards from Lake Constance, Germany,
Jens

-- 
Jens Erat

 [phone]: tel:+49-151-56961126
  [mail]: mailto:em...@jenserat.de
[jabber]: xmpp:jab...@jenserat.de
   [web]: http://www.jenserat.de

 OpenPGP: 0D69 E11F 12BD BA07 7B37  26AB 4E1F 799A A4FF 2279



smime.p7s
Description: S/MIME Cryptographic Signature

Re: [basex-talk] Getting element counts in the visualizations?

[Steinar Bang]

> Christian Grün :

>> Could it be something to do with the namespacing in the document?
> If your document has namespaces, you can use a wildcard for your prefix…

> • count(//*:A/*), count(//*:B/*)
> • for $c in /*:Top/* return count($c/*)

> or define the prefix in the query prolog:

>   declare namespace abc = 'http:...';
>   /abc:Top

Yup, I found this out at work today (without access to the email I'm
using on this list).

The document has a default namespace, and this worked fine:
 declare default element namespace 
"http://www.kith.no/xmlstds/eresept/m30/2013-10-08;;
 for $c in /FEST/* return count($c/*)

> If you use count(), the result will be a number, which will be
> displayed in the textual result view. If you return nodes, they will
> also be highlighted in the visualizations.

I wanted the element names together with their child counts, so I
improved the query to this (which made for easily paste-able org-mode
and Jira comment tables):
 declare default element namespace 
"http://www.kith.no/xmlstds/eresept/m30/2013-10-08;;
 for $c in /FEST/* return concat("|", node-name($c), "|", count($c/*), "|")

>> The actual documents I'm working on, are of the same type as the one in
>> this zip file http://goo.gl/ULH089

> Google tells me that your "goo.gl shortlink has been disabled. It was
> found to be violating our Terms of Service."…

So I also found out at work today, goo.gl shortlinks can't be used to
link to zip files.

Hopefully dropbox doesn't have this limitation:
 https://www.dropbox.com/s/p0gy01j0mfsp7ne/M30_Fest250Rekvirent_20151015.zip

Thanks for your help!

And thanks for BaseX!  The more I use it, the more I like it! :-)

Re: [basex-talk] Sending Bytes, Not Strings, To BaseX Using the Ruby Client

[Eliot Kimber]

I suspect that using the REST API directly is the answer, although if I
understand the Ruby client code, it's using direct socket connections,
which would be more efficient.

Not a critical issue at the moment but something I need to understand more
fully before too long.

Cheers,

E.

Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 2/19/16, 9:31 AM, "Christian Grün"  wrote:

>> Hmm. Is there a way to send other encodings to the server via the remote
>> API?
>
>A difficult one for me to answer, because I have never worked with
>Ruby before… Maybe there are some other users on the list who can
>reply on this?
>
>> I'm on my way to Japan for a workshop where we'll be using my system and
>> Japanese-language documents are more efficiently stored in UTF-16 so my
>> expectation is that users will either already have documents in that
>> encoding or will create new ones. Of course, for the workshop we can
>>limit
>> ourselves to UTF-8 but I'm trying to make the system as foolproof as
>> possible.
>
>Sounds interesting, and absolutely reasonable. Maybe our HTTP services
>(e.g. the default REST API) could be an alternative?
>
>Christian
>
>
>> I think the issue with my script was that I was putting quotes around
>>the
>> XML strings, which causes the server to treat it as a file path rather
>> than as XML to load. Once I fixed that then I was able to delete and add
>> files from my Ruby git hooks.
>>
>> I'll have to get a better understanding of how Ruby handles arbitrary
>>byte
>> sequences (this is where there's a little too much magic for my taste)
>>but
>> I would expect that if I provide the remote API with a byte sequence
>>that
>> starts with 0xFFFE, 0xFEFF, 0x003C003F, or 0x3C003F00 that it would
>>treat
>> it as UTF-16.
>>
>> Cheers,
>>
>> E.
>> 
>> Eliot Kimber, Owner
>> Contrext, LLC
>> http://contrext.com
>>
>>
>>
>>
>> On 2/18/16, 4:58 PM, "Christian Grün"  wrote:
>>
>>>Hi Eliot,
>>>
>>>For most client bindings, files must indeed be sent in UTF-8, so I
>>>guess it’s also the case for the Ruby binding. If the sent bytes are
>>>correct UTF-8, everything should work be fine.
>>>
>>>Christian
>>>
>>>
>>>On Thu, Feb 18, 2016 at 6:08 PM, Eliot Kimber 
>>>wrote:
 This test document as a non-ascii character '〺' (\u303A), which I
added
to
 test handling of multi-byte characters.

 Ruby and the BaseX client seem to be handling the UTF-8 correctly but
 UTF-16 didn't. I'm guessing it's Ruby's fault because it's treating
the
 bytes as a string and of course that's not going to work in a naive
way.

 Cheers,

 E.
 
 Eliot Kimber, Owner
 Contrext, LLC
 http://contrext.com




 On 2/18/16, 11:04 AM, "Eliot Kimber"
  wrote:

>I turned my UTF-8 file into a UTF-16 file and trying to commit it to
>BaseX
>via the Ruby client it did not work:
>
>BaseXClient.rb:50:in `execute': Resource "/opt/basex/?" not found.
>(RuntimeError)
>
>Where "?" is some kind of "unrecognized character" indicator
>
>Cheers,
>
>E.
>
>
>
>Eliot Kimber, Owner
>Contrext, LLC
>http://contrext.com
>
>
>
>
>On 2/18/16, 10:26 AM, "Eliot Kimber"
>ekim...@contrext.com> wrote:
>
>>I'm implementing server-side git hooks for use in GitLab under Docker
>>where Java is not available (at least that I can see). The hooks load
>>or
>>delete files from databases in BaseX.
>>
>>I'm trying to implement the hooks in Ruby (which is much more
>>pleasant
>>than bash scripting in any case) and I'm using the BaseXClient.rb
>>from
>>https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby
>>
>>I need to create or replace files by sending the bytes--I'd rather
>>not
>>read the input file into a Ruby string and send that since I don't
>>trust
>>Ruby to not hose up the data (even when it's UTF-8 I still don't
>>trust
>>it,
>>but I only started using Ruby yesterday so maybe my mistrust is
>>misplaced?).
>>
>>Using the AddExample.rb as guide, I'm doing this:
>>
>>(Earlier code to open or create database, which works).
>>
>>file = File.new("../../" + path, "rb")
>>bytes = file.read
>>file.close
>>puts "file=/#{bytes}/"
>>@basex.add(path, "#{bytes}")
>>
>>I also tried:
>>
>>@basex.add(path, bytes)
>>
>>
>>
>>And I get this result (I added some debugging messages to sendCmd()):
>>
>>ensureDatabase(): Checking database "_dfst^metadata^temp^master"...
>>BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53
>>ms.

Re: [basex-talk] Sending Bytes, Not Strings, To BaseX Using the Ruby Client

[Christian Grün]

> Hmm. Is there a way to send other encodings to the server via the remote
> API?

A difficult one for me to answer, because I have never worked with
Ruby before… Maybe there are some other users on the list who can
reply on this?

> I'm on my way to Japan for a workshop where we'll be using my system and
> Japanese-language documents are more efficiently stored in UTF-16 so my
> expectation is that users will either already have documents in that
> encoding or will create new ones. Of course, for the workshop we can limit
> ourselves to UTF-8 but I'm trying to make the system as foolproof as
> possible.

Sounds interesting, and absolutely reasonable. Maybe our HTTP services
(e.g. the default REST API) could be an alternative?

Christian


> I think the issue with my script was that I was putting quotes around the
> XML strings, which causes the server to treat it as a file path rather
> than as XML to load. Once I fixed that then I was able to delete and add
> files from my Ruby git hooks.
>
> I'll have to get a better understanding of how Ruby handles arbitrary byte
> sequences (this is where there's a little too much magic for my taste) but
> I would expect that if I provide the remote API with a byte sequence that
> starts with 0xFFFE, 0xFEFF, 0x003C003F, or 0x3C003F00 that it would treat
> it as UTF-16.
>
> Cheers,
>
> E.
> 
> Eliot Kimber, Owner
> Contrext, LLC
> http://contrext.com
>
>
>
>
> On 2/18/16, 4:58 PM, "Christian Grün"  wrote:
>
>>Hi Eliot,
>>
>>For most client bindings, files must indeed be sent in UTF-8, so I
>>guess it’s also the case for the Ruby binding. If the sent bytes are
>>correct UTF-8, everything should work be fine.
>>
>>Christian
>>
>>
>>On Thu, Feb 18, 2016 at 6:08 PM, Eliot Kimber 
>>wrote:
>>> This test document as a non-ascii character '〺' (\u303A), which I added
>>>to
>>> test handling of multi-byte characters.
>>>
>>> Ruby and the BaseX client seem to be handling the UTF-8 correctly but
>>> UTF-16 didn't. I'm guessing it's Ruby's fault because it's treating the
>>> bytes as a string and of course that's not going to work in a naive way.
>>>
>>> Cheers,
>>>
>>> E.
>>> 
>>> Eliot Kimber, Owner
>>> Contrext, LLC
>>> http://contrext.com
>>>
>>>
>>>
>>>
>>> On 2/18/16, 11:04 AM, "Eliot Kimber"
>>> >> ekim...@contrext.com> wrote:
>>>
I turned my UTF-8 file into a UTF-16 file and trying to commit it to
BaseX
via the Ruby client it did not work:

BaseXClient.rb:50:in `execute': Resource "/opt/basex/?" not found.
(RuntimeError)

Where "?" is some kind of "unrecognized character" indicator

Cheers,

E.



Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 2/18/16, 10:26 AM, "Eliot Kimber"
 wrote:

>I'm implementing server-side git hooks for use in GitLab under Docker
>where Java is not available (at least that I can see). The hooks load
>or
>delete files from databases in BaseX.
>
>I'm trying to implement the hooks in Ruby (which is much more pleasant
>than bash scripting in any case) and I'm using the BaseXClient.rb from
>https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby
>
>I need to create or replace files by sending the bytes--I'd rather not
>read the input file into a Ruby string and send that since I don't
>trust
>Ruby to not hose up the data (even when it's UTF-8 I still don't trust
>it,
>but I only started using Ruby yesterday so maybe my mistrust is
>misplaced?).
>
>Using the AddExample.rb as guide, I'm doing this:
>
>(Earlier code to open or create database, which works).
>
>file = File.new("../../" + path, "rb")
>bytes = file.read
>file.close
>puts "file=/#{bytes}/"
>@basex.add(path, "#{bytes}")
>
>I also tried:
>
>@basex.add(path, bytes)
>
>
>
>And I get this result (I added some debugging messages to sendCmd()):
>
>ensureDatabase(): Checking database "_dfst^metadata^temp^master"...
>BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53
>ms.
>Added or modified file: "test-newname.xml"
>file=/This is a test 20
>/
>
>*** sendCmd():
>cmd=
>arg=test-newname.xml
>input=This is a test 20
>BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1):
>Premature end of file. (RuntimeError)
>
>  from commit-hooks/git/server-side/BaseXClient.rb:64:in `add'
>  from commit-hooks/git/server-side/post-receive:80:in `block in
>update'
>  from commit-hooks/git/server-side/post-receive:74:in `each'
>  from commit-hooks/git/server-side/post-receive:74:in `update'
>  from 

Re: [basex-talk] Sending Bytes, Not Strings, To BaseX Using the Ruby Client

[Eliot Kimber]

Hmm. Is there a way to send other encodings to the server via the remote
API?

I'm on my way to Japan for a workshop where we'll be using my system and
Japanese-language documents are more efficiently stored in UTF-16 so my
expectation is that users will either already have documents in that
encoding or will create new ones. Of course, for the workshop we can limit
ourselves to UTF-8 but I'm trying to make the system as foolproof as
possible.

I think the issue with my script was that I was putting quotes around the
XML strings, which causes the server to treat it as a file path rather
than as XML to load. Once I fixed that then I was able to delete and add
files from my Ruby git hooks.

I'll have to get a better understanding of how Ruby handles arbitrary byte
sequences (this is where there's a little too much magic for my taste) but
I would expect that if I provide the remote API with a byte sequence that
starts with 0xFFFE, 0xFEFF, 0x003C003F, or 0x3C003F00 that it would treat
it as UTF-16.

Cheers,

E.

Eliot Kimber, Owner
Contrext, LLC
http://contrext.com




On 2/18/16, 4:58 PM, "Christian Grün"  wrote:

>Hi Eliot,
>
>For most client bindings, files must indeed be sent in UTF-8, so I
>guess it’s also the case for the Ruby binding. If the sent bytes are
>correct UTF-8, everything should work be fine.
>
>Christian
>
>
>On Thu, Feb 18, 2016 at 6:08 PM, Eliot Kimber 
>wrote:
>> This test document as a non-ascii character '〺' (\u303A), which I added
>>to
>> test handling of multi-byte characters.
>>
>> Ruby and the BaseX client seem to be handling the UTF-8 correctly but
>> UTF-16 didn't. I'm guessing it's Ruby's fault because it's treating the
>> bytes as a string and of course that's not going to work in a naive way.
>>
>> Cheers,
>>
>> E.
>> 
>> Eliot Kimber, Owner
>> Contrext, LLC
>> http://contrext.com
>>
>>
>>
>>
>> On 2/18/16, 11:04 AM, "Eliot Kimber"
>> > ekim...@contrext.com> wrote:
>>
>>>I turned my UTF-8 file into a UTF-16 file and trying to commit it to
>>>BaseX
>>>via the Ruby client it did not work:
>>>
>>>BaseXClient.rb:50:in `execute': Resource "/opt/basex/?" not found.
>>>(RuntimeError)
>>>
>>>Where "?" is some kind of "unrecognized character" indicator
>>>
>>>Cheers,
>>>
>>>E.
>>>
>>>
>>>
>>>Eliot Kimber, Owner
>>>Contrext, LLC
>>>http://contrext.com
>>>
>>>
>>>
>>>
>>>On 2/18/16, 10:26 AM, "Eliot Kimber"
>>>>>ekim...@contrext.com> wrote:
>>>
I'm implementing server-side git hooks for use in GitLab under Docker
where Java is not available (at least that I can see). The hooks load
or
delete files from databases in BaseX.

I'm trying to implement the hooks in Ruby (which is much more pleasant
than bash scripting in any case) and I'm using the BaseXClient.rb from
https://github.com/BaseXdb/basex/tree/master/basex-api/src/main/ruby

I need to create or replace files by sending the bytes--I'd rather not
read the input file into a Ruby string and send that since I don't
trust
Ruby to not hose up the data (even when it's UTF-8 I still don't trust
it,
but I only started using Ruby yesterday so maybe my mistrust is
misplaced?).

Using the AddExample.rb as guide, I'm doing this:

(Earlier code to open or create database, which works).

file = File.new("../../" + path, "rb")
bytes = file.read
file.close
puts "file=/#{bytes}/"
@basex.add(path, "#{bytes}")

I also tried:

@basex.add(path, bytes)



And I get this result (I added some debugging messages to sendCmd()):

ensureDatabase(): Checking database "_dfst^metadata^temp^master"...
BaseXResult: Database '_dfst^metadata^temp^master' was opened in 1.53
ms.
Added or modified file: "test-newname.xml"
file=/This is a test 20
/

*** sendCmd():
cmd=
arg=test-newname.xml
input=This is a test 20
BaseXClient.rb:110:in `sendCmd': "test-newname.xml.xml" (Line 1):
Premature end of file. (RuntimeError)

  from commit-hooks/git/server-side/BaseXClient.rb:64:in `add'
  from commit-hooks/git/server-side/post-receive:80:in `block in
update'
  from commit-hooks/git/server-side/post-receive:74:in `each'
  from commit-hooks/git/server-side/post-receive:74:in `update'
  from commit-hooks/git/server-side/post-receive:111:in `block in
'
  from commit-hooks/git/server-side/post-receive:103:in `each'
  from commit-hooks/git/server-side/post-receive:103:in `'
Eliots-MBP:hooks ekimber$

A couple of things here:


Where is the extra ".xml" in the target filename coming from?

What is causing the premature end of file? It feels like it's trying
interpret the second argument as a 

Re: [basex-talk] Setting base-dir for xquery:eval

[Marc van Grootel]

Wow. You guys don't stop to amaze. Chris's suggestion would've worked.
Andy's suggestion was what I actually had in mind when I wrote the
post.
So he reads my mind, Chris adds a new feature and all that before the
clock of 10 less than 12 hours after the post. And that, probably,
with a good night's sleep in between.

Of course I'll share what I have. I am creating it to better provide
testable documentation and examples for Origami (I'm still working on
it but work has kinda taken over the last few months).

Cheers,
--Marc


On Fri, Feb 19, 2016 at 9:51 AM, Christian Grün
 wrote:
>> a base-uri set. I wonder if there is a case for adding   base-uri  to the
>> xquery:eval options map to handle this in a cleaner way?
>
> …now there is [1,2]. New features in the documentation will be marked
> with 8.5, but they will already available in the snapshots and patch
> versions.
>
> [1] http://docs.basex.org/wiki/XQuery_Module#xquery:eval
> [2] http://files.basex.org/releases/latest/
>
>
>
>> On 19 Feb 2016 07:10, "Marc van Grootel"  wrote:
>>>
>>> Yes, exactly what I was after. Thanks and good night :)
>>>
>>> --Marc
>>>
>>> > On 18 feb. 2016, at 23:38, Christian Grün 
>>> > wrote:
>>> >
>>> > Hi Marc,
>>> >
>>> >> when running xquery:eval with a string it will try to resolve paths
>>> >> relative to the code module and not relative to the file the string
>>> >> came from.
>>> >
>>> > You could add a base-uri declaration in your query:
>>> >
>>> >  let $uri := 'a/b/c'
>>> >  let $query := '1'
>>> >  return xquery:eval(
>>> >'declare base-uri "' || $uri || '"; ' || $query
>>> >  )
>>> >
>>> > Does this help?
>>> > Christian
>>> >
>>> >
>>> >> Ideally I would like to be able to set the base-dir for
>>> >> xquery:eval. The asciidoc file isn't always in the correct path
>>> >> relative to the module that I need to import.
>>> >>
>>> >> Other options I'm considering but are less ideal.
>>> >>
>>> >> - Manipulate the module import paths in the query string using regexps
>>> >> - Writing out to a temp file and then use xquery:parse-uri (but this
>>> >> is tricky as I may not want to clutter existing directories with
>>> >> tempfiles only to have the imports work).
>>> >>
>>> >> Unless there are better options I think I'll opt for string massaging
>>> >> using regexps. But maybe someone has a better idea.
>>> >>
>>> >> --Marc



-- 
--Marc

Re: [basex-talk] Setting base-dir for xquery:eval

[Christian Grün]

> a base-uri set. I wonder if there is a case for adding   base-uri  to the
> xquery:eval options map to handle this in a cleaner way?

…now there is [1,2]. New features in the documentation will be marked
with 8.5, but they will already available in the snapshots and patch
versions.

[1] http://docs.basex.org/wiki/XQuery_Module#xquery:eval
[2] http://files.basex.org/releases/latest/



> On 19 Feb 2016 07:10, "Marc van Grootel"  wrote:
>>
>> Yes, exactly what I was after. Thanks and good night :)
>>
>> --Marc
>>
>> > On 18 feb. 2016, at 23:38, Christian Grün 
>> > wrote:
>> >
>> > Hi Marc,
>> >
>> >> when running xquery:eval with a string it will try to resolve paths
>> >> relative to the code module and not relative to the file the string
>> >> came from.
>> >
>> > You could add a base-uri declaration in your query:
>> >
>> >  let $uri := 'a/b/c'
>> >  let $query := '1'
>> >  return xquery:eval(
>> >'declare base-uri "' || $uri || '"; ' || $query
>> >  )
>> >
>> > Does this help?
>> > Christian
>> >
>> >
>> >> Ideally I would like to be able to set the base-dir for
>> >> xquery:eval. The asciidoc file isn't always in the correct path
>> >> relative to the module that I need to import.
>> >>
>> >> Other options I'm considering but are less ideal.
>> >>
>> >> - Manipulate the module import paths in the query string using regexps
>> >> - Writing out to a temp file and then use xquery:parse-uri (but this
>> >> is tricky as I may not want to clutter existing directories with
>> >> tempfiles only to have the imports work).
>> >>
>> >> Unless there are better options I think I'll opt for string massaging
>> >> using regexps. But maybe someone has a better idea.
>> >>
>> >> --Marc

Re: [basex-talk] Getting element counts in the visualizations?

[Christian Grün]

> Could it be something to do with the namespacing in the document?

If your document has namespaces, you can use a wildcard for your prefix…

• count(//*:A/*), count(//*:B/*)
• for $c in /*:Top/* return count($c/*)

or define the prefix in the query prolog:

  declare namespace abc = 'http:...';
  /abc:Top

If you use count(), the result will be a number, which will be
displayed in the textual result view. If you return nodes, they will
also be highlighted in the visualizations.

> The actual documents I'm working on, are of the same type as the one in
> this zip file http://goo.gl/ULH089

Google tells me that your "goo.gl shortlink has been disabled. It was
found to be violating our Terms of Service."…