Re: [Dspace-devel] Async releases discussion
On 5 May 2011 22:48, Tim Donohue wrote: > I will state that I *completely* agree that we should have an "open > development workspace" for anyone (committers or non-committers) to > create their own Maven projects & DSpace plugins. But, I think that is > *NOT* the same as what the "modules workspace" currently is. The > "modules workspace" is currently an unfortunate jumble of "stable, > controlled" modules, unstable/experimental modules, third-party > developed modules, and outdated/obsolete modules. > I admit, I'm still hazy on whether you are suggesting that we should > remove the Committer boundaries/gatekeeping on *all* Maven projects, or > just on a sub-set of those projects. Obviously, there's a big > difference there. So, I'd be curious to hear which "boundaries" you > would see as remaining, and which would "go away" based on your > suggestions. > I'm going to jump in with a fairly definitive statement / viewpoint - people can shoot it down as they see fit. There are plenty of open development workspaces that people can use for creating addons - start a project on SourceForge, Google Code, GitHub. It's trivial for anyone to just go ahead and do that, and applying their own governance, without needing to be let in the door by one of our gatekeepers. Saying that we need to provide an open development workspace is tantamount to saying - if you want to find an addon, go and look in the source tree. I've said it before, and I'll reiterate it here - that is a fundamentally broken way for people to discover what addons exist for DSpace. We should be concentrating on a user (not just developer) friendly website - directories, tagging, project pages for the addons. You discover the addons in the catalog, and the project page points to where the source is. Or, if we really want to push, lets stop screwing about with svn reorgs, granular permissions, module directories, etc. - and just run our own FusionForge server. But all these discussions and confusion are showing me is that we should just simplify it even further beyond release vs addons (which is itself a simplification from current situation of some parts of the release being in 'modules' alongside addons, prototypes, etc.)... and get back down to 'release' - and possibly a few 'important' addons - that sit in the official source tree, and are subject to all that is entailed by DSpace governance. And we absolutely encourage there to be more addons by the community, and in doing so we ensure that we are providing ways for them to be discoverable by the community - but the source tree(s) kept clearly separate from 'DSpace' itself. This would encounter an issue about moving projects between source trees when they become 'official' or 'unofficial' - but I wouldn't expect that situation to arise all that often. It sounds like a tough point of view, but really it's about giving freedom to the community. It sets addons free of the central gatekeepers, and let's them define their own path. But we can only achieve that if we draw an absolutely clear line for what is covered by DSpace governance and responsibility. > We want to attain a build process that allows for addons to exist > > independently of the release cycle. thats always been the case. > > I think we found out yesterday that several people are not sure if we > actually want that. We had several "undecided" votes (more 'undecided' > than 'decided' votes in fact). > I'll be bold and say that we probably more decided than you suggest if you take the statement of face value. Yes, if something is an addon - something that an end user / developer can take and apply to their own DSpace installation, and where we don't make a dependency on it from what we release as a "packaged" DSpace - then it can be simply be released whenever it's 'ready', at some point after the release(s) that it works with, then I don't think any committer is perturbed by it having an independent release cycle. But where we do have a dependency from the "packaged" DSpace - whether it's enabled (-stats?) or disabled (-discovery?) by default - then it ceases to be an addon. Then we do have an issue about it's release cycle, it's release procedures, it's governance, and it's accountability. And to touch back on what was an earlier point: >These are honest questions. The Committers as a whole have never >discussed these sorts of policy changes. Obviously, we *don't* allow >anyone who wants to work on XMLUI or JSPUI (as examples) to just start >committing code. These policies can always be changed, but we need to >discuss these changes before jumping to conclusions. The interesting thing here is that any UI we could ultimately treat as an 'addon' - in that it's entirely feasible (with a bit of work on things like configuration) to take either JSPUI or XMLUI - or develop any alternative UI - as an addon, which is released on a separate schedule, has it's own development team, governance, etc. Clearly, we need to have a
Re: [Dspace-devel] Async releases discussion
On Thu, May 05, 2011 at 04:48:52PM -0500, Tim Donohue wrote: [snip] > On 5/5/2011 3:43 PM, Mark Diggory wrote: [Quoting Tim Donohue, IIRC] > >> To be honest, I worry that jumping directly into technology > >> implementation ideas (git/github, etc.) only muddles things a bit. > >> I don't mean any offense with this suggestion, I'm just noting that > >> I don't think we're even all on the "same page" yet -- which is > >> where we need to be before we can make decisions on *how* we want > >> to move forward. > > > > I don't really have any questions about this, its relatively clear > > that gatekeeping cripples participation and we should try to avoid it > > as to not raise the bar to participation too high. > > I'm all for changing some of the 'gatekeeping' to make it easier for > participation. But, we do have to realize one of the main purposes of > "gatekeeping" is to ensure that each DSpace release is well vetted, > tested and approved by a "trusted" group. This is what our community of > 1000+ institutions expects from the Committers Group -- we are here to > ensure we are providing a consistently stable, well vetted & useful product. > > We can definitely change where the "gatekeeping" occurs, but I think > there will always need to be some gates in place in order to properly > 'vet' the "trustworthy" modules from the "not-so-trustworthy" ones (or > the ones that may be highly unstable). We also need to ensure the final > "packaged" DSpace product is vetted & well tested itself, so there are > additional "gates" that still need to be in place to ensure unstable or > unapproved code isn't in that "packaged" release. Agree. We want more code that is rugged, reliable, well-behaved, and of a piece with the rest of the product. "Many eyes make all bugs shallow." Don't think of it as "gatekeeping", but as "editing". An editor's job is to get the best stuff that belongs in his publication and to help writers understand what "belongs in this publication" means. A few contributions just don't belong; many can be improved by suggestions from one whose focus is the publication as a whole. Good editors don't just reject stuff; they work with promising contributors to make something better than either one of them could have done alone. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Asking whether markets are efficient is like asking whether people are smart. pgpI5Co2CqkLQ.pgp Description: PGP signature -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
Re: [Dspace-devel] OAI-ORE Harvesting & BITSTREAMS
We are doing precisely the same thing, and experiencing exactly the same results. I just discovered this in the process of migrating to 1.7.1. In my case, this method is known to work correctly in 1.6.2. James Russell OhioLINK From: pa...@jobim.org [mailto:pa...@jobim.org] Sent: Saturday, April 30, 2011 2:40 PM To: amas...@lib-gw.tamu.edu; dspace-devel@lists.sourceforge.net; dspace-gene...@lists.sourceforge.net Subject: [Dspace-devel] OAI-ORE Harvesting & BITSTREAMS Dear Alex I am having difficulties displaying the thumbnails from a collection harvested with the OAI-ORE interface using only the links to bitstreams. Whenever I read the ORE.xml document with the links to bitstream, the whole XSL system gets lost, displaying only the first line of a summaryList. I have attached a patch to the Reference theme so you can reproduce the problem. I hope you can help me Paulo Jobim -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Reopened: (DS-171) Embargo_on_Bitstream_(JSP)
[ https://jira.duraspace.org/browse/DS-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Diggory reopened DS-171: - Assignee: Mark Diggory I would like to resurrect this Issue because we have started working on an Embargo solution that actually uses the start/end dates of resource policies to enforce the embargo rather than a cron tab script. The new approach allows for Embargo to be applied at either the Item level or individual Bitstream levels and does not require executing a cronjob to adjust the state of the Item. The AuthorizationManager already supports the enforcement of timeframes in resourcePolicies. I would like to extend ResourcePolicy itself to be a better DSpace Domain Model citizen and that would include having name and description fields available to define the reason for a resource policy being set. As we evolve the Metadata capabilities to support system/tech/admin/descirptive metadata sections for all parts of the item, we can consider that the ResourcePolices will inform the production of metadata about the embargo state of the Item being exposed in OAI / SWORD /METS packagers and so-on. But for now, we really need to make the Resource Policies be the mechanism that enforces the policies within the system and not some metadata field set in the item metadata description. Mark > Embargo_on_Bitstream_(JSP) > -- > > Key: DS-171 > URL: https://jira.duraspace.org/browse/DS-171 > Project: DSpace > Issue Type: Improvement > Components: JSPUI >Reporter: Charles Kiplagat >Assignee: Mark Diggory >Priority: Major > > Bitstreams all have a resource policy. Most of the time this is a anonymous > read policy. If a bitstream has no anonymous read policy then it cannot be > read, so effectively it is under embargo.Now if you could set an end date for > that embargo, it would be so much nicer. But this is already possible: DSpace > allows resource policies to have a start date and an end date: so if you set > an anonymous read policy with a start date of januari 1 2009, it is > effectively under embargo until januari 1 2009. Now all this change does, is: > * allow a submitter to set an embargo date when uploading a new file > (optional) * allow an administrator to set the start and end dates of a > resource policy for bitstreams * display the embargo date in a nice way when > displaying an item with bitstream information, and disabling the link to that > bitstream if it is under embargo (an administrator can always link to the > bitstream) > https://openaccess.leidenuniv.nl/handle/1887/12291 > http://wiki.dspace.org/index.php/Embargo_on_Bitstream_(JSP) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Updated: (DS-171) Embargo_on_Bitstream_(JSP)
[ https://jira.duraspace.org/browse/DS-171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Diggory updated DS-171: Component/s: XMLUI Unit Testing Framework SWORD REST API (experimental) OAI-PMH DSpace API Documentation Fix Version/s: 1.8.0 > Embargo_on_Bitstream_(JSP) > -- > > Key: DS-171 > URL: https://jira.duraspace.org/browse/DS-171 > Project: DSpace > Issue Type: Improvement > Components: Documentation, DSpace API, JSPUI, OAI-PMH, REST API > (experimental), SWORD, Unit Testing Framework, XMLUI >Reporter: Charles Kiplagat >Assignee: Mark Diggory >Priority: Major > Fix For: 1.8.0 > > > Bitstreams all have a resource policy. Most of the time this is a anonymous > read policy. If a bitstream has no anonymous read policy then it cannot be > read, so effectively it is under embargo.Now if you could set an end date for > that embargo, it would be so much nicer. But this is already possible: DSpace > allows resource policies to have a start date and an end date: so if you set > an anonymous read policy with a start date of januari 1 2009, it is > effectively under embargo until januari 1 2009. Now all this change does, is: > * allow a submitter to set an embargo date when uploading a new file > (optional) * allow an administrator to set the start and end dates of a > resource policy for bitstreams * display the embargo date in a nice way when > displaying an item with bitstream information, and disabling the link to that > bitstream if it is under embargo (an administrator can always link to the > bitstream) > https://openaccess.leidenuniv.nl/handle/1887/12291 > http://wiki.dspace.org/index.php/Embargo_on_Bitstream_(JSP) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Commented: (DS-638) check files on input for viruses, and verify file format
[ https://jira.duraspace.org/browse/DS-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20217#action_20217 ] Mark H. Wood commented on DS-638: - I agree that workflow seems like a logical place to do this. What I was trying to say (and this definitely wants a ticket of its own) is that there's no need for sites to reconfigure for this if we rework ingestion so that *every submission enters a workflow*. There can be mandatory workflow steps with no user group assigned, which are carried out without human contact, such as ingestion-time curation activities (such as virus scanning). If no interactive steps are configured then that's all that happens. > check files on input for viruses, and verify file format > - > > Key: DS-638 > URL: https://jira.duraspace.org/browse/DS-638 > Project: DSpace > Issue Type: New Feature > Components: JSPUI >Affects Versions: 1.6.2 > Environment: to use this patch you will need to have ClamAV, and > jhove installed on your system. >Reporter: Jose Blanco >Assignee: Robin Taylor > Attachments: java_files.zip, jhove_config_files.zip, jsp_files.zip, > UploadStep.java, virus_check.patch > > > This patch uses JHOVE to provide rough-and-ready format checking by > identifying that the file/bitstream extension matches formats verifiable by > JHOVE. (Currently DSpace accepts a deposit's file extension as gospel, so a > user could tack a ".txt" extension onto a GIF and DSpace would assign the > incorrect format to the file based on that incorrect extension.) > This patch also also contains code to check the file for the presence of > viruses. > In order to use this patch you must have jhove and ClamAV installed on your > system. > Important notes: > (1) HTML identification has proved unreliable ( by jhove ), so this patch > does not return accurate results for that > file format. > (2) This code does not fully incorporate JHOVE's validation functions; it > only verifies that what depositors intended to submit is in fact what they > submitted. > The following are returned messages when an error is detected: > Text in [brackets] is a returned value, ALLCAPS can/should be modified to > reflect your current installation. > Questionable AIFF, GIF, JPG, PDF, TIF, WAVE, XML: > DSPACE could not verify that your file is a valid [file_format_extension]. > Please check the file format and ".[file_format_extension]" extension. > Questionable TXT: > DSPACE found the text file you are trying to upload is neither UTF-8 nor > ASCII. Please verify that your file is in the format you wanted. > Spaces in filenames ( this is an additional check ): > The file name contains spaces; this is not recommended. If possible, please > replace spaces with underscores: "_". > Virus detected: > DSPACE detected a virus in this file. Please repair it and resume the > deposit. If you need assistance, please contact us: EMAIL_ADDRESS. > To get the patch working: > Add the jhove conf files to > [dspace]/jhove direcoty > Here are the conf files: > jhove-aiff.conf > jhove-ascii.conf > jhove-gif.conf > jhove-jpeg.conff > jhove-pdf.conf > jhove-tiff.conf > jhove-utf8.conf > jhove-wave.conf > jhove-xml.conf > Also the following files were changed: > dspace-api/src/main/java/org/dspace/submit/step/UploadStep.java > dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/submit/step/JSPUploadStep.java > dspace-api/src/main/java/org/dspace/content/FormatIdentifier.java > dspace/modules/jspui/src/main/webapp/submit/get-file-format.jsp ( locally > customized ) > dspace/modules/jspui/src/main/webapp/submit/upload-error-virus.jsp ( new file > - placed in locally modified area for the jspui interface) > These files are attached with this patch. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
[Dspace-devel] [DuraSpace JIRA] Commented: (DS-638) check files on input for viruses, and verify file format
[ https://jira.duraspace.org/browse/DS-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=20218#action_20218 ] Mark Diggory commented on DS-638: - Actually, the point of putting it in the file upload step is so the submitter can correct and be aware of the problem on the file that is causing it. It shouldn't even get into the Bitstream, the file should be tested prior to addition to DSpace. Most of the Webapps now cache the upload in a file object behind the HttpRServletReqest object or inside cocoon. The point of doing a virus scan in file upload is so the user knows theres a problem and fixes it before it ever reaches DSpace or the Reviewer Workflow. Also Configurable Reviewer workflow will certainly be able to support automated steps that do things like Jove detection etc > check files on input for viruses, and verify file format > - > > Key: DS-638 > URL: https://jira.duraspace.org/browse/DS-638 > Project: DSpace > Issue Type: New Feature > Components: JSPUI >Affects Versions: 1.6.2 > Environment: to use this patch you will need to have ClamAV, and > jhove installed on your system. >Reporter: Jose Blanco >Assignee: Robin Taylor > Attachments: java_files.zip, jhove_config_files.zip, jsp_files.zip, > UploadStep.java, virus_check.patch > > > This patch uses JHOVE to provide rough-and-ready format checking by > identifying that the file/bitstream extension matches formats verifiable by > JHOVE. (Currently DSpace accepts a deposit's file extension as gospel, so a > user could tack a ".txt" extension onto a GIF and DSpace would assign the > incorrect format to the file based on that incorrect extension.) > This patch also also contains code to check the file for the presence of > viruses. > In order to use this patch you must have jhove and ClamAV installed on your > system. > Important notes: > (1) HTML identification has proved unreliable ( by jhove ), so this patch > does not return accurate results for that > file format. > (2) This code does not fully incorporate JHOVE's validation functions; it > only verifies that what depositors intended to submit is in fact what they > submitted. > The following are returned messages when an error is detected: > Text in [brackets] is a returned value, ALLCAPS can/should be modified to > reflect your current installation. > Questionable AIFF, GIF, JPG, PDF, TIF, WAVE, XML: > DSPACE could not verify that your file is a valid [file_format_extension]. > Please check the file format and ".[file_format_extension]" extension. > Questionable TXT: > DSPACE found the text file you are trying to upload is neither UTF-8 nor > ASCII. Please verify that your file is in the format you wanted. > Spaces in filenames ( this is an additional check ): > The file name contains spaces; this is not recommended. If possible, please > replace spaces with underscores: "_". > Virus detected: > DSPACE detected a virus in this file. Please repair it and resume the > deposit. If you need assistance, please contact us: EMAIL_ADDRESS. > To get the patch working: > Add the jhove conf files to > [dspace]/jhove direcoty > Here are the conf files: > jhove-aiff.conf > jhove-ascii.conf > jhove-gif.conf > jhove-jpeg.conff > jhove-pdf.conf > jhove-tiff.conf > jhove-utf8.conf > jhove-wave.conf > jhove-xml.conf > Also the following files were changed: > dspace-api/src/main/java/org/dspace/submit/step/UploadStep.java > dspace-jspui/dspace-jspui-api/src/main/java/org/dspace/app/webui/submit/step/JSPUploadStep.java > dspace-api/src/main/java/org/dspace/content/FormatIdentifier.java > dspace/modules/jspui/src/main/webapp/submit/get-file-format.jsp ( locally > customized ) > dspace/modules/jspui/src/main/webapp/submit/upload-error-virus.jsp ( new file > - placed in locally modified area for the jspui interface) > These files are attached with this patch. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://jira.duraspace.org/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
