[MarkLogic Dev General] XDMP-INMTRPLIDXFULL
I noticed similar log entries to this on our 7.0-3 cluster and I am not sure if it is anything to be concerned with- 2014-07-28 00:23:00.812 Debug: Forest::insert: Meters XDMP-INMTRPLIDXFULL: In-memory triple-index storage full; list: table=2%, wordsused=2%, wordsfree=97%, overhead=1%; tree: table=1%, wordsused=11%, wordsfree=89%, overhead=0% We aren't using any triple indexes in any of the databases on the system. Any info/advice is appreciated. Harry ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL
Hi Harry, MarkLogics built-in history logging (the Meters database) is using triples. So, nothing to immediately worry about. It might affect monitoring though. I'd have to check what could be behind this.. Cheers, Geert Van: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] Namens Harry Bakken Verzonden: maandag 28 juli 2014 12:08 Aan: General MarkLogic Developer Discussion Onderwerp: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL I noticed similar log entries to this on our 7.0-3 cluster and I am not sure if it is anything to be concerned with- 2014-07-28 00:23:00.812 Debug: Forest::insert: Meters XDMP-INMTRPLIDXFULL: In-memory triple-index storage full; list: table=2%, wordsused=2%, wordsfree=97%, overhead=1%; tree: table=1%, wordsused=11%, wordsfree=89%, overhead=0% We aren't using any triple indexes in any of the databases on the system. Any info/advice is appreciated. Harry ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL
This is a Debug level log so that itself is an indication that this is not an issue but or even generally 'informative' for normal use (or it would be a warning or higher). Its due to the metering collection and its indicating that enough memory was used to that the in memory Index was flushed to disk. From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Geert Josten Sent: Monday, July 28, 2014 6:32 AM To: 'MarkLogic Developer Discussion' Subject: Re: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL Hi Harry, MarkLogics built-in history logging (the Meters database) is using triples. So, nothing to immediately worry about. It might affect monitoring though. I'd have to check what could be behind this.. Cheers, Geert Van: general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] Namens Harry Bakken Verzonden: maandag 28 juli 2014 12:08 Aan: General MarkLogic Developer Discussion Onderwerp: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL I noticed similar log entries to this on our 7.0-3 cluster and I am not sure if it is anything to be concerned with- 2014-07-28 00:23:00.812 Debug: Forest::insert: Meters XDMP-INMTRPLIDXFULL: In-memory triple-index storage full; list: table=2%, wordsused=2%, wordsfree=97%, overhead=1%; tree: table=1%, wordsused=11%, wordsfree=89%, overhead=0% We aren't using any triple indexes in any of the databases on the system. Any info/advice is appreciated. Harry ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] Security Design
I am in the early design stages of a (hopefully) large application and would like to see if I understand the operations of collections correctly. You can think of this in a similar context to a social media app. I have attached a simple diagram to aid the text. Imagine that Joe, Sue and Tom are users and each have a collection (marked 'P' )where only they have read/write access to documents they load. Joe and Tom have collections that they would like to use to share (read only) with various other users, one being Sue. This seems rather straight forward. However, the use case also calls for Sue being able to share (read only) Tom's documents with Joe and Joe's documents with Tom; as she sees fit without the intervention of Tom or Joe. Could someone expand on this to describe how this might be setup? Do I need separate roles that are tied to each collection, for each of these exchanges? Thanks, Tim -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.org ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
[MarkLogic Dev General] XSLT check if result document has been created
Hi All
I’m working on an xslt transform where I’m extracting data about images from a
document and put
that data into result document, It all works fine except for when the same
image occurs more than once as I then get conflicting uris,
my code looks like this
xsl:template match=//db:informalfigure[descendant::db:imagedata and
@role='figure']
xsl:variable name=curImage
select=substring-after(.//@fileref,'/')/
xsl:variable name=id
select=$imageMetaData//image[name=string-join(($curISBN,$curImage),'/')]/id/
xsl:if test=not(doc-available(string-join(('out/',$id,'.xml'),'')))
!--This check fails, am I doing it incorrectly or is it the way xslt
processes document that makes it hard to check if the id have been encountered
before?--
xsl:result-document method=xml href=out/{$id}.xml indent=yes
Ideally I would like to be able to check if the result document have been
created or not and after that decide if I want to update it with more
information or just leave it be.
Would appreciate any help on the subject
Best regards
Erik
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XSLT check if result document has been created
I think you may be running afoul of URI resolution.
Since the URI you are giving to doc-available is a relative
URI, it will be resolved relative to the static base URI, which
per XSLT is the URI of the stylesheet itself.
//Mary
On Mon, 28 Jul 2014 07:58:04 -0700, Erik Zander
erik.zan...@studentlitteratur.se wrote:
Hi All
I’m working on an xslt transform where I’m extracting data about images
from a document and put
that data into result document, It all works fine except for when the
same image occurs more than once as I then get conflicting uris,
my code looks like this
xsl:template match=//db:informalfigure[descendant::db:imagedata and
@role='figure']
xsl:variable name=curImage
select=substring-after(.//@fileref,'/')/
xsl:variable name=id
select=$imageMetaData//image[name=string-join(($curISBN,$curImage),'/')]/id/
xsl:if
test=not(doc-available(string-join(('out/',$id,'.xml'),'')))
!--This check fails, am I doing it incorrectly or is it the way xslt
processes document that makes it hard to check if the id have been
encountered before?--
xsl:result-document method=xml href=out/{$id}.xml
indent=yes
Ideally I would like to be able to check if the result document have
been created or not and after that decide if I want to update it with
more information or just leave it be.
Would appreciate any help on the subject
Best regards
Erik
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XSLT check if result document has been created
I believe you should be able to do something like the following to get what
you want:
xsl:template match=//db:informalfigure[descendant::db:imagedata and
@role='figure']
xsl:variable name=curImage select=
substring-after(.//@fileref,'/')/
xsl:variable name=id select=
$imageMetaData//image[name=string-join(($curISBN,$curImage),'/')]/id/
xsl:if test=not(doc-available(
xdmp:resolve-uri(string-join(('out/',$id,'.xml'),''),xdmp:node-uri(.
!--This check fails, am I doing it incorrectly or is it the way xslt
processes document that makes it hard to check if the id have been
encountered before?--
xsl:result-document method=xml href=out/{$id}.xml indent=
yes
On Mon, Jul 28, 2014 at 9:08 AM, Mary Holstege mary.holst...@marklogic.com
wrote:
I think you may be running afoul of URI resolution.
Since the URI you are giving to doc-available is a relative
URI, it will be resolved relative to the static base URI, which
per XSLT is the URI of the stylesheet itself.
//Mary
On Mon, 28 Jul 2014 07:58:04 -0700, Erik Zander
erik.zan...@studentlitteratur.se wrote:
Hi All
I’m working on an xslt transform where I’m extracting data about images
from a document and put
that data into result document, It all works fine except for when the
same image occurs more than once as I then get conflicting uris,
my code looks like this
xsl:template match=//db:informalfigure[descendant::db:imagedata and
@role='figure']
xsl:variable name=curImage
select=substring-after(.//@fileref,'/')/
xsl:variable name=id
select=$imageMetaData//image[name=string-join(($curISBN,$curImage),'/')]/id/
xsl:if
test=not(doc-available(string-join(('out/',$id,'.xml'),'')))
!--This check fails, am I doing it incorrectly or is it the way xslt
processes document that makes it hard to check if the id have been
encountered before?--
xsl:result-document method=xml href=out/{$id}.xml
indent=yes
Ideally I would like to be able to check if the result document have
been created or not and after that decide if I want to update it with
more information or just leave it be.
Would appreciate any help on the subject
Best regards
Erik
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
___
General mailing list
General@developer.marklogic.com
http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Security Design
Hi Tim, I am not sure I have thought this through completely, nor can I think of the exact steps to do this, but here is my instinct on how I would attempt to solve this: I would try to create amped functions that allow Sue to share (read only) Tom’s document (that Sue has read permission for). I think the function could amp to a role paired with a read permission on the document, thus allowing Tom to read the document. Like I said, I am not totally sure how I would write such a function, but it seems possible (though tricky). See if that scratches an itch. Maybe someone else has a better idea. -Danny From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy W. Cook Sent: Monday, July 28, 2014 7:19 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Security Design I am in the early design stages of a (hopefully) large application and would like to see if I understand the operations of collections correctly. You can think of this in a similar context to a social media app. I have attached a simple diagram to aid the text. Imagine that Joe, Sue and Tom are users and each have a collection (marked 'P' )where only they have read/write access to documents they load. Joe and Tom have collections that they would like to use to share (read only) with various other users, one being Sue. This seems rather straight forward. However, the use case also calls for Sue being able to share (read only) Tom's documents with Joe and Joe's documents with Tom; as she sees fit without the intervention of Tom or Joe. Could someone expand on this to describe how this might be setup? Do I need separate roles that are tied to each collection, for each of these exchanges? Thanks, Tim -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.orghttp://www.mlhim.org/ ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Security Design
In the builtin security model, simply adding a collection to a document won't make that document visible to another user. Visibility of documents is controlled by read permissions and roles. Granting read permission for a document to a role makes that document visible to users that have that role. Using the builtin security model, each user would have its own role, and the default permissions would include insert, read, and update for that role. Users could also grant permissions to other user roles. However that might not be optimal for some of your most common use-cases. First, re-sharing could get tricky. Re-sharing a document by changing its permissions requires updating it. That may be the trickiest part of your implementation, because if I have update permission I can make any other changes too. That might lead you to use an amp'd function for resharing, and you might have to tracking sharing state outside the builtin security model. Adding some document properties might take care of that. Also it's not clear to me how your users will think of these collections. But let's say they look and act like folders or directories. With dropbox, for example, I can share a specific document or an entire folder. If I create a new document in a shared folder, that's also shared. That would work if you enable inherited permissions using the built-in security model, or using extra code in your upload function. However automatic inheritance requires directory fragments. But enabling automatic directory creation can be a bottleneck for concurrent updates. Maybe you could get away with directory-creation=manual, and call xdmp:directory-create only when sharing a directory. Or implement your own inheritance code that runs whenever a new document is created, and maintain your own documents that record sharing relationships. Extending that idea, another approach is to maintain your own pseudo-security model. This is sometimes called entitlements. This is closer to how dropbox works, and it has pros and cons. You have more flexibility, but any bugs in your code risk revealing documents that ought not to be shared. In this model you would maintain your own documents that record sharing relationships. When requesting a document or directory you'd use that sharing information to build a query using cts:document-query, cts:directory-query, and perhaps even cts:collection-query. The query results will only include documents that the user is authorized to see. This is very similar to how the built-in security model works, but using your own data model rather than directory fragments. -- Mike On 28 Jul 2014, at 07:18 , Timothy W. Cook t...@mlhim.org wrote: I am in the early design stages of a (hopefully) large application and would like to see if I understand the operations of collections correctly. You can think of this in a similar context to a social media app. I have attached a simple diagram to aid the text. Imagine that Joe, Sue and Tom are users and each have a collection (marked 'P' )where only they have read/write access to documents they load. Joe and Tom have collections that they would like to use to share (read only) with various other users, one being Sue. This seems rather straight forward. However, the use case also calls for Sue being able to share (read only) Tom's documents with Joe and Joe's documents with Tom; as she sees fit without the intervention of Tom or Joe. Could someone expand on this to describe how this might be setup? Do I need separate roles that are tied to each collection, for each of these exchanges? Thanks, Tim -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.org Collections.png___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Security Design
Thanks Danny. I was hoping to get a discussion around this because I am not sure it is as simple as I first thought. I had not considered using amped functions to do it. Of course this was just a three person example. The real use case involves Sue needing to have the ability to share read access to several other users on documents where she only has read access. At this point I have minimal exposure to the way MarkLogic handles document level security and trying to relate this to the documentation in the Search Guide on Collections security. So, I think this needs good planning before jumping into implementation. A specific question is; do I understand correctly that a document can be in many collections and that collection access must be granted before document access is checked? --Tim On Mon, Jul 28, 2014 at 7:31 PM, Danny Sokolsky danny.sokol...@marklogic.com wrote: Hi Tim, I am not sure I have thought this through completely, nor can I think of the exact steps to do this, but here is my instinct on how I would attempt to solve this: I would try to create amped functions that allow Sue to share (read only) Tom’s document (that Sue has read permission for). I think the function could amp to a role paired with a read permission on the document, thus allowing Tom to read the document. Like I said, I am not totally sure how I would write such a function, but it seems possible (though tricky). See if that scratches an itch. Maybe someone else has a better idea. -Danny *From:* general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] *On Behalf Of *Timothy W. Cook *Sent:* Monday, July 28, 2014 7:19 AM *To:* MarkLogic Developer Discussion *Subject:* [MarkLogic Dev General] Security Design I am in the early design stages of a (hopefully) large application and would like to see if I understand the operations of collections correctly. You can think of this in a similar context to a social media app. I have attached a simple diagram to aid the text. Imagine that Joe, Sue and Tom are users and each have a collection (marked 'P' )where only they have read/write access to documents they load. Joe and Tom have collections that they would like to use to share (read only) with various other users, one being Sue. This seems rather straight forward. However, the use case also calls for Sue being able to share (read only) Tom's documents with Joe and Joe's documents with Tom; as she sees fit without the intervention of Tom or Joe. Could someone expand on this to describe how this might be setup? Do I need separate roles that are tied to each collection, for each of these exchanges? Thanks, Tim -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.org ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.org ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] Security Design
For your question about collections, there are 2 types of collections, “collections” and “protected collections”. Collections have no security associated with them, and a document can have many (or no) collections, but access to the document is simply by permissions. Protected collections are a bit different, and in general, I would not recommend using them. They create a document in the security database, and they only affect access to the documents when asking for them *by collection* (ie, using fn:collection). When accessing a document by a protected collection, you need to have permissions on both the collection and the document. -Danny From: general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy W. Cook Sent: Monday, July 28, 2014 3:52 PM To: MarkLogic Developer Discussion Subject: Re: [MarkLogic Dev General] Security Design Thanks Danny. I was hoping to get a discussion around this because I am not sure it is as simple as I first thought. I had not considered using amped functions to do it. Of course this was just a three person example. The real use case involves Sue needing to have the ability to share read access to several other users on documents where she only has read access. At this point I have minimal exposure to the way MarkLogic handles document level security and trying to relate this to the documentation in the Search Guide on Collections security. So, I think this needs good planning before jumping into implementation. A specific question is; do I understand correctly that a document can be in many collections and that collection access must be granted before document access is checked? --Tim On Mon, Jul 28, 2014 at 7:31 PM, Danny Sokolsky danny.sokol...@marklogic.commailto:danny.sokol...@marklogic.com wrote: Hi Tim, I am not sure I have thought this through completely, nor can I think of the exact steps to do this, but here is my instinct on how I would attempt to solve this: I would try to create amped functions that allow Sue to share (read only) Tom’s document (that Sue has read permission for). I think the function could amp to a role paired with a read permission on the document, thus allowing Tom to read the document. Like I said, I am not totally sure how I would write such a function, but it seems possible (though tricky). See if that scratches an itch. Maybe someone else has a better idea. -Danny From: general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com [mailto:general-boun...@developer.marklogic.commailto:general-boun...@developer.marklogic.com] On Behalf Of Timothy W. Cook Sent: Monday, July 28, 2014 7:19 AM To: MarkLogic Developer Discussion Subject: [MarkLogic Dev General] Security Design I am in the early design stages of a (hopefully) large application and would like to see if I understand the operations of collections correctly. You can think of this in a similar context to a social media app. I have attached a simple diagram to aid the text. Imagine that Joe, Sue and Tom are users and each have a collection (marked 'P' )where only they have read/write access to documents they load. Joe and Tom have collections that they would like to use to share (read only) with various other users, one being Sue. This seems rather straight forward. However, the use case also calls for Sue being able to share (read only) Tom's documents with Joe and Joe's documents with Tom; as she sees fit without the intervention of Tom or Joe. Could someone expand on this to describe how this might be setup? Do I need separate roles that are tied to each collection, for each of these exchanges? Thanks, Tim -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.orghttp://www.mlhim.org/ ___ General mailing list General@developer.marklogic.commailto:General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general -- Timothy Cook LinkedIn Profile:http://www.linkedin.com/in/timothywaynecook MLHIM http://www.mlhim.orghttp://www.mlhim.org/ ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
Re: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL
Awesome, thanks. On Jul 28, 2014 5:06 AM, David Lee david@marklogic.com wrote: This is a Debug level log so that itself is an indication that this is not an issue but or even generally 'informative' for normal use (or it would be a warning or higher). Its due to the metering collection and its indicating that enough memory was used to that the in memory Index was flushed to disk. *From:* general-boun...@developer.marklogic.com [mailto: general-boun...@developer.marklogic.com] *On Behalf Of *Geert Josten *Sent:* Monday, July 28, 2014 6:32 AM *To:* 'MarkLogic Developer Discussion' *Subject:* Re: [MarkLogic Dev General] XDMP-INMTRPLIDXFULL Hi Harry, MarkLogics built-in history logging (the Meters database) is using triples. So, nothing to immediately worry about. It might affect monitoring though. I'd have to check what could be behind this.. Cheers, Geert *Van:* general-boun...@developer.marklogic.com [ mailto:general-boun...@developer.marklogic.com general-boun...@developer.marklogic.com] *Namens *Harry Bakken *Verzonden:* maandag 28 juli 2014 12:08 *Aan:* General MarkLogic Developer Discussion *Onderwerp:* [MarkLogic Dev General] XDMP-INMTRPLIDXFULL I noticed similar log entries to this on our 7.0-3 cluster and I am not sure if it is anything to be concerned with- 2014-07-28 00:23:00.812 Debug: Forest::insert: Meters XDMP-INMTRPLIDXFULL: In-memory triple-index storage full; list: table=2%, wordsused=2%, wordsfree=97%, overhead=1%; tree: table=1%, wordsused=11%, wordsfree=89%, overhead=0% We aren't using any triple indexes in any of the databases on the system. Any info/advice is appreciated. Harry ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general ___ General mailing list General@developer.marklogic.com http://developer.marklogic.com/mailman/listinfo/general
