Re: FW: [dspace-tech] cocoon error, item view

[wlrutherford]

After reconfiguring our statistics and restarting DSpace I'm seeing the 
same sort of error as Susan Borda in the cocoon log:
  2018-11-02 00:01:15,820 
WARN  org.apache.cocoon.components.xslt.TraxErrorListener  - Can not load 
requested doc: unknown protocol: cocoon at 
file:///dspace/webapps/ROOT/themes/Mirage/lib/xsl/core/page-structure.xsl:547:130

Most of the messages are just informational and seem to be due to the 
system catching up on statistics. To be safe I'll disable
our cron scripts which run every night to update stats until whatever the 
system is doing is complete.

The other errors we're seeing look like this:
  2018-11-02 00:05:25,935 WARN  cocoon.access  - 
org.apache.cocoon.ConnectionResetException: Connection reset by peer

Has anyone else seen these errors and, more importantly, fixed the problem?

Thank you,

  Walter R.


On Monday, February 15, 2016 at 10:11:02 PM UTC-9, Germán Biozzoli wrote:
>
> Hi people
>
> I'm about to resurrect this message from Susan.
>
> We're exactly at the same point, but in our case seems to occur some 
> obscure problem relative to the authorization of the items. The "blank" 
> items were public and now the collection is restricted to a group of users. 
> Using the advanced authorization tool, we defined READ rights for the 
> group/item/collection and DEFAULT BITSTREAM READ for 
> group/bistream/collection. After that, no posibility to view the items, 
> neither administrator or some e-person from the group. Adding ADMIN, WRITE 
> and other rights does not make any effect, the only way to continue viewing 
> metadata is deleting all policies over the items.
>
> The error message is the same and restating Tomcat neither produces any 
> effect.
>
> It happens in a dev instance using 6.0-SNAPSHOT.
>
> Any ideas where to look for the error?
>
> Regards
> Germán
>
>
>  
>
>  
>
> 2015-12-15 18:30 GMT-03:00 Borda, Susan  >:
>
>> Hi All-
>> We never received a response to this question and unfortunately it
>> continues to happen, again this morning for instance and once again a
>> restart of Tomcat resolved the issue.
>>
>> Has anyone else had this problem?
>>
>> We are now running DSpace 5.4.
>>
>> Any advice would be great.
>>
>> Thank you!
>> susan
>>
>> ‹
>> Susan Borda
>> Digital Technologies Development Librarian
>> Montana State University Library
>> 406-994-1873
>>
>>
>>
>>
>>
>>
>>
>> On 11/5/15, 2:51 PM, "dspac...@googlegroups.com  on behalf 
>> of Erik Guss"
>>  on behalf of 
>> eg...@auth.lib.montana.edu >
>> wrote:
>>
>> >Dear Community:
>> >
>> >particulars: CentOS6.7, tomcat7, dspace5.3, mirage2 xmlui, <10,000 items
>> >
>> >Today our DSpace was displaying an intermittent error. All functionality
>> >worked except the View item. In other words, you could browse around,
>> >search, and see hitlists. But when clicking on a handle's item view, the
>> >menus would appear, but no item content. This condition was fixed by
>> >restarting tomcat. I've included log snippets from cocoon and dspace.log
>> >below, first from the unsuccessful item view at 12:19 PM, then from the
>> >successful item view after restarting tomcat at 12:49 PM. The problem
>> >seems to have something to do with the cache. At the time of the
>> >problem, there were no postgresql errors, or catalina.out errors, and
>> >top showed normal resource usage.
>> >
>> >**cocoon.log**
>> >** item information is blank on the screen during this time frame 12:19 
>> **
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache HIT for
>>
>> >PK_G-aspect-cocoon://DRI/12/handle/1/9083?pipelinehash=-398319662474980475
>> >5
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>>
>> >PK_G-aspect-cocoon://DRI/11/handle/1/9083_T-Navigation-8967015258021785944
>> >_T-ItemViewer--1872229765608651772
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>>
>> >PK_G-aspect-cocoon://DRI/10/handle/1/9083_T-Navigation-8967015258021785944
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>> >PK_G-aspect-cocoon://DRI/9/handle/1/9083_T-Navigation-8967015258021785944
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>>
>> >PK_G-aspect-cocoon://DRI/8/handle/1/9083_T-Include-file:///usr/local/dspac
>> >e/config/MSU-sidebar-menu.xml
>> >2015-11-05 12:19:22,468 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>>
>> >PK_G-aspect-cocoon://DRI/7/handle/1/9083_T-SystemwideAlerts-1_T-Navigation
>> >--5367372201529954577
>> >2015-11-05 12:19:22,469 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for
>> >PK_G-aspect-cocoon://DRI/6/handle/1/9083_T-Navigation-5799876941772906259
>> >2015-11-05 12:19:22,469 INFO  org.apache.cocoon.caching.impl.CacheImpl  -
>> >Cache MISS for PK_G-aspect-cocoon://DRI/5/handle/1/9083
>> >2015-11-05 12:19:22,469 INFO  

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Sometimes you have lucky accidents. When the upgrade of httpd failed I 
uninstalled httpd and mod_ssl and
reinstalled them both. Not only did httpd start with the new version but it 
gave me an error I hadn't seen before:

[root@dspace36a tmp]# service httpd restart
Stopping httpd:[  OK  ]
Starting httpd: [Mon Oct 29 12:10:46 2018] [warn] _default_ VirtualHost 
overlap on port 443, the first has precedence
   [  OK  ]

Ah-ha!  There was a conflict between my vhosts.conf file and one of the 
Include files that configures SSL.
I stripped out the redundancies and overlaps and now the site is secured 
for ports 80 and 443 as expected.
Port 8080 is not secured but it was suggested that one unsecured port might 
be needed for OAI so I'll leave
it like that.

In general should I have a virtual host configuration for port 8080 also or 
does it just fall through unchanged?

Thanks for you help!



On Monday, October 29, 2018 at 11:04:30 AM UTC-8, wlruth...@alaska.edu 
wrote:
>
> Frustrated and starting fresh Monday morning. I commented out ALL of the 
> virtual host configurations in
> my vhosts.conf file for ports 80 and 443; nothing remains. So, anything 
> that goes through httpd will thus
> fall through the file untouched. Sure enough, as I expected, port 8080 
> continued to work just fine. So the
> problem must be in the httpd setup someplace.
>
> When I saw nothing obvious in the configuration I went back to the 
> DSpace5.5 manual and might've found
> the culprit. There is a warning that Apache 2.4.2 (or lower) breaks the 
> connection to the backend (Tomcat).
> I'd seen that warning before but, since it just said "Apache" (and I 
> wasn't using httpd at the time) I thought it
> meant Apache Tomcat, so all was fine. Today I checked our version of 
> httpd; we are running Apache/2.2.15.
> Updating httpd via yum gave the same version of httpd, just one compiled 
> in Feb 2018 rather than Jul 2017.
> If that didn't fix the problem I'll obtain the source code and compile a 
> version  of httpd newer than v2.4.2.
>
> Stay tuned...
>
> Nope, that broke it so now even port 8080 doesn't work. Sigh... Looking 
> for source code for Apache httpd.
>
>
>
>
> On Friday, October 26, 2018 at 1:23:45 PM UTC-8, wlruth...@alaska.edu 
> wrote:
>>
>> Yes, it does make sense. It also means I must have something 
>> misconfigured because the only way I can
>> currently access the site is if I explicitly specify http and port 8080 - 
>> http://dspace36a.library.uaf.edu:8080/
>>
>>
>> On Friday, October 26, 2018 at 12:37:42 PM UTC-8, Tim Donohue wrote:
>>
>> Hi Walter,
>>
>> No worries, we've all been newbies at some point :)
>>
>> Port 8080 is Tomcat. It should not be defined in Apache configuration.  
>> Think of it this way, when you use Apache in front of Tomcat... your Tomcat 
>> ports do *not* need to be publicly accessible. They are just internal ports 
>> (only accessible on localhost).
>>
>> So, in the setup I described above, my *only public ports* are port 80 
>> and 443.  I don't have port 8080 or port 8009 available publicly, so you 
>> cannot access my site via http://my.url:8080/ (for example).  Port 8080 
>> and 8009 are only accessible on the server (so they are private ports).  
>> So, if I login to the server and do a "wget http://localhost:8080; then 
>> Tomcat responds.  And Port 8009 is just used for proxying requests between 
>> Apache and Tomcat on that server.
>>
>> So, with regards to SSL, in my setup the only port where SSL is 
>> configured is port 443.  As I have port 80 (HTTP) requests redirecting to 
>> port 443 (HTTPS), *all* public requests (except sometimes OAI, as 
>> mentioned) go through SSL. *After* SSL is validated, those requests *all* 
>> get proxied to Tomcat (via port 8009).  Port 8009 and 8080 don't need SSL 
>> (again, cause they are not public ports and cannot be accessed unless you 
>> first login directly on the server).
>>
>> Does that start to make a bit more sense?
>>
>> Tim
>>
>> On Fri, Oct 26, 2018 at 3:28 PM  wrote:
>>
>> Good to know. That should simplify things.
>>
>> Now for the embarrassing newb questions. I'm still not clear how my httpd 
>> secures port 8080
>> (which everything is going through now) since a URL that includes port 
>> 8080 isn't handled by
>> the port 443 virtual host configuration. And how does port 8009 (AJP 
>> Tomcat) get involved;
>> it also is only referenced in the port 80 and 443 configurations. Your 
>> example didn't show any-
>> configuration for port 8080 which otherwise should just fall thru the 
>> other virtual hosts checks.
>>
>> I'm pretty sure I still have something misconfigured since our port 8080 
>> refuses any https
>> connection attempts, so I think SSL is being completely ignored. From 
>> your earlier post httpd
>> should be handling all connection attempts first then passing control to 
>> Tomcat. I added some
>> dummy rewrites 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Frustrated and starting fresh Monday morning. I commented out ALL of the 
virtual host configurations in
my vhosts.conf file for ports 80 and 443; nothing remains. So, anything 
that goes through httpd will thus
fall through the file untouched. Sure enough, as I expected, port 8080 
continued to work just fine. So the
problem must be in the httpd setup someplace.

When I saw nothing obvious in the configuration I went back to the 
DSpace5.5 manual and might've found
the culprit. There is a warning that Apache 2.4.2 (or lower) breaks the 
connection to the backend (Tomcat).
I'd seen that warning before but, since it just said "Apache" (and I wasn't 
using httpd at the time) I thought it
meant Apache Tomcat, so all was fine. Today I checked our version of httpd; 
we are running Apache/2.2.15.
Updating httpd via yum gave the same version of httpd, just one compiled in 
Feb 2018 rather than Jul 2017.
If that didn't fix the problem I'll obtain the source code and compile a 
version  of httpd newer than v2.4.2.

Stay tuned...

Nope, that broke it so now even port 8080 doesn't work. Sigh... Looking for 
source code for Apache httpd.




On Friday, October 26, 2018 at 1:23:45 PM UTC-8, wlruth...@alaska.edu wrote:
>
> Yes, it does make sense. It also means I must have something misconfigured 
> because the only way I can
> currently access the site is if I explicitly specify http and port 8080 - 
> http://dspace36a.library.uaf.edu:8080/
>
>
> On Friday, October 26, 2018 at 12:37:42 PM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> No worries, we've all been newbies at some point :)
>
> Port 8080 is Tomcat. It should not be defined in Apache configuration.  
> Think of it this way, when you use Apache in front of Tomcat... your Tomcat 
> ports do *not* need to be publicly accessible. They are just internal ports 
> (only accessible on localhost).
>
> So, in the setup I described above, my *only public ports* are port 80 and 
> 443.  I don't have port 8080 or port 8009 available publicly, so you cannot 
> access my site via http://my.url:8080/ (for example).  Port 8080 and 8009 
> are only accessible on the server (so they are private ports).  So, if I 
> login to the server and do a "wget http://localhost:8080; then Tomcat 
> responds.  And Port 8009 is just used for proxying requests between Apache 
> and Tomcat on that server.
>
> So, with regards to SSL, in my setup the only port where SSL is configured 
> is port 443.  As I have port 80 (HTTP) requests redirecting to port 443 
> (HTTPS), *all* public requests (except sometimes OAI, as mentioned) go 
> through SSL. *After* SSL is validated, those requests *all* get proxied to 
> Tomcat (via port 8009).  Port 8009 and 8080 don't need SSL (again, cause 
> they are not public ports and cannot be accessed unless you first login 
> directly on the server).
>
> Does that start to make a bit more sense?
>
> Tim
>
> On Fri, Oct 26, 2018 at 3:28 PM  wrote:
>
> Good to know. That should simplify things.
>
> Now for the embarrassing newb questions. I'm still not clear how my httpd 
> secures port 8080
> (which everything is going through now) since a URL that includes port 
> 8080 isn't handled by
> the port 443 virtual host configuration. And how does port 8009 (AJP 
> Tomcat) get involved;
> it also is only referenced in the port 80 and 443 configurations. Your 
> example didn't show any-
> configuration for port 8080 which otherwise should just fall thru the 
> other virtual hosts checks.
>
> I'm pretty sure I still have something misconfigured since our port 8080 
> refuses any https
> connection attempts, so I think SSL is being completely ignored. From your 
> earlier post httpd
> should be handling all connection attempts first then passing control to 
> Tomcat. I added some
> dummy rewrites to the port 80 and 443 vhost definitions that redirect to a 
> different site if the
> URI matches "/catalog"; they are both definitely catching and handling 
> URLs for those ports.
>
> Do you also have a virtual host configuration for port 8080 and does it 
> also Proxy via 8009?
>
> Thank you,
>
>   Walter R.
>
>
> On Friday, October 26, 2018 at 10:17:46 AM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> The dspace.cfg settings (any that use a URL) do not *require* a port 
> number.  The port number only needs to be specified if it's a non-standard 
> port (e.g. 8080).  So, if you are using port 80 or 443, those can be left 
> off of any configuration as long as you appropriately use http:// or 
> https:// where necessary.
>
>
> - Tim
>
> On Thu, Oct 25, 2018 at 6:04 PM  wrote:
>
> Thanks, that does help clarify a lot.
>
> I already have port 80 forwarding everything to port 443 and I saw that 
> the AJP port was already setup by default in server.xml.
> I just hadn't seen in the DSpace manual where I needed it and I wasn't 
> sure what it was used for. All that I'm missing is a virtual
> host entry like you use to redirect (most) connections to pass thru to 
> Tomcat AJP. We've 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Yes, it does make sense. It also means I must have something misconfigured 
because the only way I can
currently access the site is if I explicitly specify http and port 8080 - 
http://dspace36a.library.uaf.edu:8080/


On Friday, October 26, 2018 at 12:37:42 PM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> No worries, we've all been newbies at some point :)
>
> Port 8080 is Tomcat. It should not be defined in Apache configuration.  
> Think of it this way, when you use Apache in front of Tomcat... your Tomcat 
> ports do *not* need to be publicly accessible. They are just internal ports 
> (only accessible on localhost).
>
> So, in the setup I described above, my *only public ports* are port 80 and 
> 443.  I don't have port 8080 or port 8009 available publicly, so you cannot 
> access my site via http://my.url:8080/ (for example).  Port 8080 and 8009 
> are only accessible on the server (so they are private ports).  So, if I 
> login to the server and do a "wget http://localhost:8080; then Tomcat 
> responds.  And Port 8009 is just used for proxying requests between Apache 
> and Tomcat on that server.
>
> So, with regards to SSL, in my setup the only port where SSL is configured 
> is port 443.  As I have port 80 (HTTP) requests redirecting to port 443 
> (HTTPS), *all* public requests (except sometimes OAI, as mentioned) go 
> through SSL. *After* SSL is validated, those requests *all* get proxied to 
> Tomcat (via port 8009).  Port 8009 and 8080 don't need SSL (again, cause 
> they are not public ports and cannot be accessed unless you first login 
> directly on the server).
>
> Does that start to make a bit more sense?
>
> Tim
>
> On Fri, Oct 26, 2018 at 3:28 PM > 
> wrote:
>
>> Good to know. That should simplify things.
>>
>> Now for the embarrassing newb questions. I'm still not clear how my httpd 
>> secures port 8080
>> (which everything is going through now) since a URL that includes port 
>> 8080 isn't handled by
>> the port 443 virtual host configuration. And how does port 8009 (AJP 
>> Tomcat) get involved;
>> it also is only referenced in the port 80 and 443 configurations. Your 
>> example didn't show any-
>> configuration for port 8080 which otherwise should just fall thru the 
>> other virtual hosts checks.
>>
>> I'm pretty sure I still have something misconfigured since our port 8080 
>> refuses any https
>> connection attempts, so I think SSL is being completely ignored. From 
>> your earlier post httpd
>> should be handling all connection attempts first then passing control to 
>> Tomcat. I added some
>> dummy rewrites to the port 80 and 443 vhost definitions that redirect to 
>> a different site if the
>> URI matches "/catalog"; they are both definitely catching and handling 
>> URLs for those ports.
>>
>> Do you also have a virtual host configuration for port 8080 and does it 
>> also Proxy via 8009?
>>
>> Thank you,
>>
>>   Walter R.
>>
>>
>> On Friday, October 26, 2018 at 10:17:46 AM UTC-8, Tim Donohue wrote:
>>
>>> Hi Walter,
>>>
>>> The dspace.cfg settings (any that use a URL) do not *require* a port 
>>> number.  The port number only needs to be specified if it's a non-standard 
>>> port (e.g. 8080).  So, if you are using port 80 or 443, those can be left 
>>> off of any configuration as long as you appropriately use http:// or 
>>> https:// where necessary.
>>>
>>
>>> - Tim
>>>
>>> On Thu, Oct 25, 2018 at 6:04 PM  wrote:
>>>
>> Thanks, that does help clarify a lot.
>>>
>>> I already have port 80 forwarding everything to port 443 and I saw that 
>>> the AJP port was already setup by default in server.xml.
>>> I just hadn't seen in the DSpace manual where I needed it and I wasn't 
>>> sure what it was used for. All that I'm missing is a virtual
>>> host entry like you use to redirect (most) connections to pass thru to 
>>> Tomcat AJP. We've tested OAI and it was working through
>>> port 443, but I'll keep that bypass in mind in case one of the other 
>>> associated campuses has an SSL incompatible setup.
>>>
>>> The instructions/comments in the default dspace.cfg file say to include 
>>> the port numbers for dspace.baseUrl and dspace.url, but
>>> your dspace.url does not include a port number. So I assume your 
>>> dspace.baseUrl also doesn't include a port number and just
>>> lets rewrite rules handle that? Or does it even matter since all roads 
>>> eventually lead to port 8009?
>>>
>>> I tried the new setup. httpd and Tomcat started but I ended up with the 
>>> same situation. Only port 8080 is responding; ports 80
>>> and 443 fail with an error:
>>>
>>>
>>>   You don't have permission to access / on this server.
>>>
>>>   Additionally, a 403 Forbidden error was encountered while trying to 
>>> use an ErrorDocument to handle the request.
>>>
>>> So I tried again with port 443 defined in dspace.url and dspace.baseUrl. 
>>> No change. Still getting the above error.
>>> So I'm still missing something. This is what my new simpler vhosts.conf 
>>> file looks like:
>>>
>>> 
>>>   

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Good to know. That should simplify things.

Now for the embarrassing newb questions. I'm still not clear how my httpd 
secures port 8080
(which everything is going through now) since a URL that includes port 8080 
isn't handled by
the port 443 virtual host configuration. And how does port 8009 (AJP 
Tomcat) get involved;
it also is only referenced in the port 80 and 443 configurations. Your 
example didn't show any-
configuration for port 8080 which otherwise should just fall thru the other 
virtual hosts checks.

I'm pretty sure I still have something misconfigured since our port 8080 
refuses any https
connection attempts, so I think SSL is being completely ignored. From your 
earlier post httpd
should be handling all connection attempts first then passing control to 
Tomcat. I added some
dummy rewrites to the port 80 and 443 vhost definitions that redirect to a 
different site if the
URI matches "/catalog"; they are both definitely catching and handling URLs 
for those ports.

Do you also have a virtual host configuration for port 8080 and does it 
also Proxy via 8009?

Thank you,

  Walter R.


On Friday, October 26, 2018 at 10:17:46 AM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> The dspace.cfg settings (any that use a URL) do not *require* a port 
> number.  The port number only needs to be specified if it's a non-standard 
> port (e.g. 8080).  So, if you are using port 80 or 443, those can be left 
> off of any configuration as long as you appropriately use http:// or 
> https:// where necessary.
>
> - Tim
>
> On Thu, Oct 25, 2018 at 6:04 PM > 
> wrote:
>
> Thanks, that does help clarify a lot.
>
> I already have port 80 forwarding everything to port 443 and I saw that 
> the AJP port was already setup by default in server.xml.
> I just hadn't seen in the DSpace manual where I needed it and I wasn't 
> sure what it was used for. All that I'm missing is a virtual
> host entry like you use to redirect (most) connections to pass thru to 
> Tomcat AJP. We've tested OAI and it was working through
> port 443, but I'll keep that bypass in mind in case one of the other 
> associated campuses has an SSL incompatible setup.
>
> The instructions/comments in the default dspace.cfg file say to include 
> the port numbers for dspace.baseUrl and dspace.url, but
> your dspace.url does not include a port number. So I assume your 
> dspace.baseUrl also doesn't include a port number and just
> lets rewrite rules handle that? Or does it even matter since all roads 
> eventually lead to port 8009?
>
> I tried the new setup. httpd and Tomcat started but I ended up with the 
> same situation. Only port 8080 is responding; ports 80
> and 443 fail with an error:
>
>
>   You don't have permission to access / on this server.
>
>   Additionally, a 403 Forbidden error was encountered while trying to use 
> an ErrorDocument to handle the request.
>
> So I tried again with port 443 defined in dspace.url and dspace.baseUrl. 
> No change. Still getting the above error.
> So I'm still missing something. This is what my new simpler vhosts.conf 
> file looks like:
>
> 
> ### Redirect insecure default port to secure port 443
> ServerAdmin uaf-libra...@alaska.edu 
> ServerName dspace36a.library.uaf.edu
> ServerAlias dspace36a
>
> # Redirect all HTTP requests to HTTPS (EXCEPT perhaps /oai/ path)
> RewriteEngine On
>#RewriteCond %{REQUEST_URI} !^/oai/
> RewriteRule ^(.*) https://%{SERVER_NAME}$1 [R=permanent,L]
>
> # Pass all requests (which weren't previously redirected to HTTPS) 
> to Tomcat's AJP Connector (on port 8009)
> ProxyPass/ ajp://localhost:8009/
> ProxyPassReverse / ajp://localhost:8009/
> 
>
> 
> ### secure httpd server listening on port 443.
>
> ServerAdmin uaf-libra...@alaska.edu 
> #DocumentRoot /dspace/webapps/xmlui
> DocumentRoot /dspace/webapps/ROOT
> ServerName dspace36a.library.uaf.edu
> ServerAlias dspace36a
>
> ErrorLog logs/dspace36a.library.uaf.edu-https-error_log
> CustomLog logs/dspace36a.library.uaf.edu-https-access_log 
> combinedssl
> RewriteLog logs/dspace36a-rewrite
>
> Include conf.d/ssl.include
> Include conf.d/ssl.include.star
> ### ^ these point to files which specify where the ssl certificates
> ###   live on the host.
>
> # Pass all remaining requests to Tomcat's AJP Connector
> ProxyPass/ ajp://localhost:8009/
> ProxyPassReverse / ajp://localhost:8009/
> 
>
> Re-verified these entries in the server.xml file: 
>   maxThreads="150"
> minSpareThreads="25"
> maxSpareThreads="75"
> acceptCount="100"
> connectionTimeout="2"
> disableUploadTimeout="true"
> URIEncoding="UTF-8"
>  />
>
> 
> 
> 
>
> I didn't see anywhere else that I needed to 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Just a bit of follow-up to my last post.

First, I was wrong - logging in as an admin didn't make the restricted 443 
pages visible.
Second, I also understand why port 8080 is working now but not 443 which 
had worked
before. Because I removed redirectPort="443" from the 
definitions in the server.xml file. All it did was move the live port. I'm 
not sure though why
that broke all of the statistics links unless they must be non-SSL.

My problems from the start might be a misunderstanding of which ports need 
to be
secured and which need to be non-SSL. I thought that was the purpose of 
port 8080.
But at least with everything going through port 8080 I can actually see the 
stats links
now along with everything else so I'm getting very close.


On Thursday, October 25, 2018 at 6:37:03 AM UTC-8, wlruth...@alaska.edu 
wrote:
>
> Thanks Tim,
>
> I added that stats rewrite section because, for the life of me, I couldn't 
> get the stats links to work.
> I don't believe it's even necessary, especially since, as you noticed, the 
> /solr link can't be reached
> from the network. But we didn't want it to be and I was still trying to 
> figure out how to block it. Once
> things are configured correctly the software restricts it. I haven't tried 
> the link while logged in as an
> admin but I was glad to see it said I didn't have authorization otherwise.
>
> We aren't currently using AJP but if it will simplify the setup then I'll 
> read up on it. Thank you.
>
>   Walter R.
>
>
> On Thursday, October 25, 2018 at 5:49:10 AM UTC-8, Tim Donohue wrote:
>>
>> Hi Walter,
>>
>> Aha, yes, I'm starting to think the issue here is in your Apache <-> 
>> Tomcat communications/redirects.  The number of redirects/rewrites is quite 
>> a bit confusing in all those files.  I think you could really simply the 
>> setup by just using "mod_proxy" and "mod_proxy_ajp", which is our 
>> recommended approach for running DSpace on Tomcat behind Apache. See the 
>> installation guide section at:
>>
>> https://wiki.duraspace.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-UsingSSLonApacheHTTPDinfrontofTomcat(runningonports80and443
>> )
>>
>> If you use AJP for the connection between Tomcat & Apache, the number of 
>> rewrites/redirects is vastly reduced.  Additionally, you can keep Solr 
>> running on Tomcat (port 8080) and *not even reference it from your Apache 
>> settings*. Solr does not need to be publicly accessible on the web, DSpace 
>> just needs to have it running on Tomcat.  As long as DSpace can access Solr 
>> on localhost, you are fine...and all the /statistics, /statistics-home, etc 
>> paths in DSpace should just work (on whatever port your DSpace runs on). 
>>
>> So, while you can continue to build your own redirects, it's gonna be a 
>> lot harder to get DSpace running consistently.  I'd highly recommend 
>> considering using AJP, as it should make your life a lot easier (and the 
>> full setup should be in the documentation link above)
>>
>> - Tim
>>
>>
>>
>> On Wed, Oct 24, 2018 at 8:55 PM  wrote:
>>
>>> I made some progress and lost a little ground at the same time. First I 
>>> verified that all of the statistics config files were
>>> watching the same port:
>>>
>>> */dspace/config/modules/solr-statistics.cfg*
>>> server = http://localhost:8080/solr/statistics
>>>
>>> */dspace/config/modules/oai.cfg*
>>> solr.url=http://localhost:8080/solr/oai
>>>
>>> */dspace/config/modules/discovery.cfg*
>>>  search.server = http://localhost:8080/solr/search
>>>
>>> */dspace/config/dspace.cfg*
>>> solr.authority.server=
>>> https://dspace36a.library.uaf.edu:8080/solr/authority
>>>  choices.plugin.dc.contributor.author = SolrAuthorAuthority
>>>  choices.presentation.dc.contributor.author = authorLookup
>>>  authority.controlled.dc.contributor.author = true
>>>
>>> # Still trying to get publicly viewable statistics. I was sure these had 
>>> been set
>>> # in the past but I can't find them in this file now. Perhaps solr 
>>> config file.  10/15/18 WLR
>>> report.public = true
>>> statistics.item.authorization.admin = false
>>>
>>> # 10/19/18 WLR // updated 10/24/18 WLR
>>> #solr.server = http://localhost:8080/solr
>>> solr.server = http://localhost:8080/solr/statistics
>>>
>>>
>>> */etc/httpd/conf.d/vhosts.conf*
>>> 
>>> ### Redirect insecure default port to secure port 443
>>> ServerAdmin uaf-libra...@alaska.edu
>>> RewriteEngine On
>>> RewriteCond %{HTTPS} off
>>> RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
>>> 
>>>
>>> 
>>> ### secure httpd server listening on port 443.
>>>
>>> ServerAdmin uaf-libra...@alaska.edu
>>> RewriteEngine On
>>> # There's probably a better way to do this
>>> RewriteCond %{REQUEST_URI} "^/solr"  
>>> [NC]
>>> RewriteCond %{REQUEST_URI} "^/statistic"
>>> [NC]
>>> RewriteCond %{REQUEST_URI} "^/statistics-home"[NC]
>>> RewriteCond 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Thanks Tim,

I added that stats rewrite section because, for the life of me, I couldn't 
get the stats links to work.
I don't believe it's even necessary, especially since, as you noticed, the 
/solr link can't be reached
from the network. But we didn't want it to be and I was still trying to 
figure out how to block it. Once
things are configured correctly the software restricts it. I haven't tried 
the link while logged in as an
admin but I was glad to see it said I didn't have authorization otherwise.

We aren't currently using AJP but if it will simplify the setup then I'll 
read up on it. Thank you.

  Walter R.


On Thursday, October 25, 2018 at 5:49:10 AM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> Aha, yes, I'm starting to think the issue here is in your Apache <-> 
> Tomcat communications/redirects.  The number of redirects/rewrites is quite 
> a bit confusing in all those files.  I think you could really simply the 
> setup by just using "mod_proxy" and "mod_proxy_ajp", which is our 
> recommended approach for running DSpace on Tomcat behind Apache. See the 
> installation guide section at:
>
> https://wiki.duraspace.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-UsingSSLonApacheHTTPDinfrontofTomcat(runningonports80and443
> )
>
> If you use AJP for the connection between Tomcat & Apache, the number of 
> rewrites/redirects is vastly reduced.  Additionally, you can keep Solr 
> running on Tomcat (port 8080) and *not even reference it from your Apache 
> settings*. Solr does not need to be publicly accessible on the web, DSpace 
> just needs to have it running on Tomcat.  As long as DSpace can access Solr 
> on localhost, you are fine...and all the /statistics, /statistics-home, etc 
> paths in DSpace should just work (on whatever port your DSpace runs on). 
>
> So, while you can continue to build your own redirects, it's gonna be a 
> lot harder to get DSpace running consistently.  I'd highly recommend 
> considering using AJP, as it should make your life a lot easier (and the 
> full setup should be in the documentation link above)
>
> - Tim
>
>
>
> On Wed, Oct 24, 2018 at 8:55 PM > 
> wrote:
>
>> I made some progress and lost a little ground at the same time. First I 
>> verified that all of the statistics config files were
>> watching the same port:
>>
>> */dspace/config/modules/solr-statistics.cfg*
>> server = http://localhost:8080/solr/statistics
>>
>> */dspace/config/modules/oai.cfg*
>> solr.url=http://localhost:8080/solr/oai
>>
>> */dspace/config/modules/discovery.cfg*
>>  search.server = http://localhost:8080/solr/search
>>
>> */dspace/config/dspace.cfg*
>> solr.authority.server=
>> https://dspace36a.library.uaf.edu:8080/solr/authority
>>  choices.plugin.dc.contributor.author = SolrAuthorAuthority
>>  choices.presentation.dc.contributor.author = authorLookup
>>  authority.controlled.dc.contributor.author = true
>>
>> # Still trying to get publicly viewable statistics. I was sure these had 
>> been set
>> # in the past but I can't find them in this file now. Perhaps solr config 
>> file.  10/15/18 WLR
>> report.public = true
>> statistics.item.authorization.admin = false
>>
>> # 10/19/18 WLR // updated 10/24/18 WLR
>> #solr.server = http://localhost:8080/solr
>> solr.server = http://localhost:8080/solr/statistics
>>
>>
>> */etc/httpd/conf.d/vhosts.conf*
>> 
>> ### Redirect insecure default port to secure port 443
>> ServerAdmin uaf-libra...@alaska.edu 
>> RewriteEngine On
>> RewriteCond %{HTTPS} off
>> RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
>> 
>>
>> 
>> ### secure httpd server listening on port 443.
>>
>> ServerAdmin uaf-libra...@alaska.edu 
>> RewriteEngine On
>> # There's probably a better way to do this
>> RewriteCond %{REQUEST_URI} "^/solr"  
>> [NC]
>> RewriteCond %{REQUEST_URI} "^/statistic"
>> [NC]
>> RewriteCond %{REQUEST_URI} "^/statistics-home"[NC]
>> RewriteCond %{REQUEST_URI} "^/search-statistics"  [NC]
>> RewriteCond %{REQUEST_URI} "^/workflow-statistics"   [NC]
>> RewriteRule (.*) http://%{HTTP_HOST}:8080%{REQUEST_URI} [R,L]
>>
>> DocumentRoot /dspace/webapps/xmlui
>> ServerName dspace36a.library.uaf.edu
>> ServerAlias scholarworks
>>
>> ErrorLog logs/dspace36a.library.uaf.edu-https-error_log
>> CustomLog logs/dspace36a.library.uaf.edu-https-access_log 
>> combinedssl
>> RewriteLog logs/dspace36a-rewrite
>>
>> Include conf.d/ssl.include
>> Include conf.d/ssl.include.star
>> ### ^ these point to a file which specifies where the ssl 
>> certificates
>> ###   live on the host.
>>
>> #   ProxyPass / http://127.0.0.1:8080/  retry=10 
>> connectiontimeout=5
>> ProxyPass /  http://127.0.0.1:443/retry=10 
>> connectiontimeout=5
>> ### ^ this tells httpd to 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

I made some progress and lost a little ground at the same time. First I 
verified that all of the statistics config files were
watching the same port:

*/dspace/config/modules/solr-statistics.cfg*
server = http://localhost:8080/solr/statistics

*/dspace/config/modules/oai.cfg*
solr.url=http://localhost:8080/solr/oai

*/dspace/config/modules/discovery.cfg*
 search.server = http://localhost:8080/solr/search

*/dspace/config/dspace.cfg*
solr.authority.server=https://dspace36a.library.uaf.edu:8080/solr/authority
 choices.plugin.dc.contributor.author = SolrAuthorAuthority
 choices.presentation.dc.contributor.author = authorLookup
 authority.controlled.dc.contributor.author = true

# Still trying to get publicly viewable statistics. I was sure these had 
been set
# in the past but I can't find them in this file now. Perhaps solr config 
file.  10/15/18 WLR
report.public = true
statistics.item.authorization.admin = false

# 10/19/18 WLR // updated 10/24/18 WLR
#solr.server = http://localhost:8080/solr
solr.server = http://localhost:8080/solr/statistics


*/etc/httpd/conf.d/vhosts.conf*

### Redirect insecure default port to secure port 443
ServerAdmin uaf-library-it-d...@alaska.edu
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}



### secure httpd server listening on port 443.

ServerAdmin uaf-library-it-d...@alaska.edu
RewriteEngine On
# There's probably a better way to do this
RewriteCond %{REQUEST_URI} "^/solr"  
[NC]
RewriteCond %{REQUEST_URI} "^/statistic"[NC]
RewriteCond %{REQUEST_URI} "^/statistics-home"[NC]
RewriteCond %{REQUEST_URI} "^/search-statistics"  [NC]
RewriteCond %{REQUEST_URI} "^/workflow-statistics"   [NC]
RewriteRule (.*) http://%{HTTP_HOST}:8080%{REQUEST_URI} [R,L]

DocumentRoot /dspace/webapps/xmlui
ServerName dspace36a.library.uaf.edu
ServerAlias scholarworks

ErrorLog logs/dspace36a.library.uaf.edu-https-error_log
CustomLog logs/dspace36a.library.uaf.edu-https-access_log 
combinedssl
RewriteLog logs/dspace36a-rewrite

Include conf.d/ssl.include
Include conf.d/ssl.include.star
### ^ these point to a file which specifies where the ssl 
certificates
###   live on the host.

#   ProxyPass / http://127.0.0.1:8080/  retry=10 connectiontimeout=5
ProxyPass /  http://127.0.0.1:443/retry=10 
connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 8080

#timeout=300

#   ProxyPassReverse /  http://127.0.0.1:8080/  retry=10
ProxyPassReverse /   http://127.0.0.1:443/retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.

#SSLUseStapling on
### Added due to error when starting apache.  Dayne Ellanna




#
### Redirect insecure ports to secure port 443
ServerAdmin uaf-library-it-d...@alaska.edu
RewriteEngine On
RewriteCond %{HTTPS} on
RewriteRule (.*) http://%{HTTP_HOST}:8080/%{REQUEST_URI} [R,L]
DocumentRoot /dspace/webapps/xmlui
ServerName dspace36a.library.uaf.edu
ServerAlias scholarworks

ErrorLog logs/dspace36a.library.uaf.edu-error_log
CustomLog logs/dspace36a.library.uaf.edu-access_log common
RewriteLog logs/dspace36a-rewrite

ProxyPass / http://127.0.0.1:8080/  retry=10 connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 443

#timeout=300
ProxyPassReverse / http://127.0.0.1:8080/  retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

#   ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.



*[tomcat]/conf/server.xml*

 

So I managed to break both ports 80 and 443 but, amazingly port 8080 works 
like a charm
and gives the statistics I've been trying to see for many weeks! I suspect 
the new problem
is in the vhosts.conf file but haven't yet caught what's going wrong.
http://dspace36a.library.uaf.edu:8080/

The stats links now work for the whole site (Home Page) and for individual 
items, although
I'm not entirely sure the single item stats are working completely 
correctly since I haven't
seen them working until now and there are often blanks within the returned 
information.

=



On Friday, October 19, 2018 at 1:10:05 PM UTC-8, wlruth...@alaska.edu wrote:
>
>
> I set 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

I set "solr.server" to use port 8080 in /dspace/config/dspace.cfg as well 
as "server" in solr-statistics.cfg and "solr.url" in oai.cfg.
Looks like progress, it still fails but the error messages changed:

2018-10-19 12:58:58,999 ERROR 
org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error 
occurred while creating statistics for home page
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: 
Expected mime type application/octet-stream but got text/html. Apache Tomcat/9.0.0.M17 - Error 
reporth1 
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
 
h2 
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;}
 
h3 
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;}
 
body 
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} b 
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} 
p 
{font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}
 
a {color:black;} a.name {color:black;} .line 
{height:1px;background-color:#525D76;border:none;} 
HTTP Status 404 - /solr/selecttype Status reportmessage 
/solr/selectdescription The requested resource is 
not available.Apache 
Tomcat/9.0.0.M17
at 
org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:512)
at 
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at 
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at 
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
...




On Friday, October 19, 2018 at 11:36:45 AM UTC-8, Sean Kalynuk wrote:
>
> Hi Walter,
>
>  
>
> Since there is redirect logic involved, how about updating your 
> solr.server parameter to point directly to the Tomcat solr webapp on port 
> 8080?
>
>  
>
> solr.server = http://localhost:8080/solr
>
>  
>
> Not entirely sure, but I’m guessing this type of change requires a Tomcat 
> restart.
>
>  
>
> --
>
> Sean
>
>  
>
> *From:* dspac...@googlegroups.com   > *On Behalf Of *wlruth...@alaska.edu 
> *Sent:* October 19, 2018 2:26 PM
> *To:* DSpace Technical Support >
> *Subject:* Re: [dspace-tech] View Statistics links not working
>
>  
>
>  
>
> Hey Sean,
>
>  
>
> I started to say I saw nothing in the logs but I just saw a redirection 
> error in dspace.log.2018-10-19 that I haven't seen before:
>
> 2018-10-19 10:51:13,345 ERROR 
> org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error 
> occurred while creating statistics for home page
>
> org.apache.solr.client.solrj.SolrServerException: Server at 
> http://localhost/statistics sent back a redirect (302).
>
> ...
>
> Rather than include the entire ~180 line stack trace, here is the 
> /etc/httpd/conf.d/vhosts.conf file which handles HTTP redirection although 
> I think
>
> we may have been having problems with the statistics links even before 
> adding this file:
>
> [root@dspace36a conf.d]# cat vhosts.conf
>
> 
>
> #
>
> ### Redirect insecure ports to secure port 443
>
> ServerAdmin uaf-library-it-d...@alaska.edu 
>
> RewriteEngine On
>
> RewriteCond %{HTTPS} off
>
> RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI} [R,L]
>
> DocumentRoot /dspace/webapps/xmlui
>
> ServerName dspace36a.library.uaf.edu
>
> ServerAlias scholarworks
>
>  
>
> ErrorLog logs/dspace36a.library.uaf.edu-error_log
>
> CustomLog logs/dspace36a.library.uaf.edu-access_log common
>
> RewriteLog logs/dspace36a-rewrite
>
>  
>
> ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
>
> ### ^ this tells httpd to redirect it's / to localhost port 443
>
>  
>
> #timeout=300
>
> ProxyPassReverse / http://127.0.0.1:443/  retry=10
>
> ### ^ this tells httpd that tomcat's url's should be rewritten to 
> look
>
> ###   like they're coming from httpd.
>
>  
>
> #   ProxyPreserveHost On
>
> ### ^ this tells httpd to keep the Host: information from the 
> client and
>
> ### pass it on to tomcat.
>
>  
>
> 
>
>  
>
> #
>
> 
>
> ### ^ this creates a httpd server that listens on port 443.
>
>  
>
> ServerAdmin uaf-library-it-d...@alaska.edu 
>
> DocumentRoot /dspace/webapps/xmlui
>
> ServerName dspace36a.library.uaf.edu
>
> ServerAlias scholarworks
>
>  
>
> ErrorLog logs/dspace36a.library.uaf.edu-https-error_log
>
> CustomLog logs/dspace36a.library.uaf.edu-https-access_log 
> combinedssl
>
> RewriteLog logs/dspace36a-rewrite
>
>  
>
> Include conf.d/ssl.include
>
> Include conf.d/ssl.include.star
>
> ### ^ these point to a file which specifies where the ssl 
> certificates
>
> ###   live on the host.
>
>  
>
> ProxyPass / 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Hey Sean,

I started to say I saw nothing in the logs but I just saw a redirection 
error in dspace.log.2018-10-19 that I haven't seen before:
2018-10-19 10:51:13,345 ERROR 
org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error 
occurred while creating statistics for home page
org.apache.solr.client.solrj.SolrServerException: Server at 
http://localhost/statistics sent back a redirect (302).
...
Rather than include the entire ~180 line stack trace, here is the 
/etc/httpd/conf.d/vhosts.conf file which handles HTTP redirection although 
I think
we may have been having problems with the statistics links even before 
adding this file:

[root@dspace36a conf.d]# cat vhosts.conf

#
### Redirect insecure ports to secure port 443
ServerAdmin uaf-library-it-d...@alaska.edu
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI} [R,L]
DocumentRoot /dspace/webapps/xmlui
ServerName dspace36a.library.uaf.edu
ServerAlias scholarworks

ErrorLog logs/dspace36a.library.uaf.edu-error_log
CustomLog logs/dspace36a.library.uaf.edu-access_log common
RewriteLog logs/dspace36a-rewrite

ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 443

#timeout=300
ProxyPassReverse / http://127.0.0.1:443/  retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

#   ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.



#

### ^ this creates a httpd server that listens on port 443.

ServerAdmin uaf-library-it-d...@alaska.edu
DocumentRoot /dspace/webapps/xmlui
ServerName dspace36a.library.uaf.edu
ServerAlias scholarworks

ErrorLog logs/dspace36a.library.uaf.edu-https-error_log
CustomLog logs/dspace36a.library.uaf.edu-https-access_log 
combinedssl
RewriteLog logs/dspace36a-rewrite

Include conf.d/ssl.include
Include conf.d/ssl.include.star
### ^ these point to a file which specifies where the ssl 
certificates
###   live on the host.

ProxyPass / http://127.0.0.1:8080/  retry=10 connectiontimeout=5
#   ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 8080

#timeout=300

ProxyPassReverse /  http://127.0.0.1:8080/  retry=10
#   ProxyPassReverse /  http://127.0.0.1:443/  retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.



Ports 80, 443, 8000, and 8080 are active for TCP. Port 8000 is used by the 
handle-server but 8080 should also be unsecured.



On Friday, October 19, 2018 at 8:46:33 AM UTC-8, Sean Kalynuk wrote:
>
> Hi Walter,
>
>  
>
> Do you see errors in the logs when visiting the following page?
>
>  
>
> https://dspace36a.library.uaf.edu/search-statistics
>
>  
>
> At least on that page I got “There was an error while generating the 
> search statistics, please try again later.”
>
>  
>
> You can also see an error message when visiting
>
>  
>
> https://dspace36a.library.uaf.edu/workflow-statistics
>
>  
>
> --
>
> Sean
>
>  
>
> *From:* dspac...@googlegroups.com   > *On Behalf Of *wlruth...@alaska.edu 
> *Sent:* October 18, 2018 2:52 PM
> *To:* DSpace Technical Support >
> *Subject:* Re: [dspace-tech] View Statistics links not working
>
>  
>
> This is maddening and taking far longer than it should, mostly because I 
> can't tell where the problem is located.
>
> I've setup all of the configuration variables mentioned in the manual but 
> I'm not seeing anything in the logfiles
>
> that says what's wrong. It could be a combination of Tomcat/DSpace, httpd, 
> solr, ... The only other similar posts
>
> I've seen here haven't had solutions. It doesn't help that I haven't see 
> the statistics links working so I'm not quite
>
> sure even what to expect.
>
>  
>
> For example: I can see https://dspace36a.library.uaf.edu/solr but that 
> looks more like an administrator page. But
>
> I don't need to be logged in at all to see it and I'm pretty sure that's 
> not the statistics page. If I try this instead:
>
> https://dspace36a.library.uaf.edu/statistics I get an entirely blank 
> page. In my experience that means it's being
>
> handled (it's not returning an error) but the output is null.
>
>  
>
> I can attach config files or logs but which would be most useful to help 
> troubleshoot this problem?
>
>  
>
>  Thank you.
>
>  
>
>Walter R.
>
>  
>
>
> On Wednesday, October 10, 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

This is maddening and taking far longer than it should, mostly because I 
can't tell where the problem is located.
I've setup all of the configuration variables mentioned in the manual but 
I'm not seeing anything in the logfiles
that says what's wrong. It could be a combination of Tomcat/DSpace, httpd, 
solr, ... The only other similar posts
I've seen here haven't had solutions. It doesn't help that I haven't see 
the statistics links working so I'm not quite
sure even what to expect.

For example: I can see https://dspace36a.library.uaf.edu/solr but that 
looks more like an administrator page. But
I don't need to be logged in at all to see it and I'm pretty sure that's 
not the statistics page. If I try this instead:
https://dspace36a.library.uaf.edu/statistics I get an entirely blank page. 
In my experience that means it's being
handled (it's not returning an error) but the output is null.

I can attach config files or logs but which would be most useful to help 
troubleshoot this problem?

 Thank you.

   Walter R.


On Wednesday, October 10, 2018 at 12:17:03 PM UTC-8, wlruth...@alaska.edu 
wrote:
>
> I am occasionally seeing errors like this in the catalina.out file:
>
> Error using query type: 2
> [WARN] deprecation - The 'component-configurations' section in the sitemap 
> is deprecated. Please check for alternatives.
> Error using query *:*
> Error using query statistics_type:workflow AND NOT(previousWorkflowStep: 
> SUBMIT)
> Error using query type: 2 AND  id:2695
> Error using query type: 0 AND owningItem:2695
> Error using query type: 2 AND  id:3677
> Error using query type: 0 AND owningItem:3677
>
> The four digit numbers appear to be valid item numbers so I'm not sure 
> what's generating the errors. So I'm slogging
> through the manual and sitemaps to see if I can trace what happens when 
> the "View Usage Statistics" link is clicked.
>
>
>
> On Tuesday, October 9, 2018 at 2:48:54 PM UTC-8, wlruth...@alaska.edu 
> wrote:
>>
>> Rats... couldn't be that easy.
>>
>> Finding very few non-INFO messages in the logs. I did find an entry in 
>> catalina.out that at least tells me it is/was running solr (perhaps with 
>> memory leaks).
>> 01-Jun-2017 01:23:38.504 WARNING [localhost-startStop-2] 
>> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The 
>> web application [solr] appears to have started a thread named 
>> [MultiThreadedHttpConnectionManager cleanu
>> p] but has failed to stop it. This is very likely to create a memory 
>> leak. Stack trace of thread:
>>
>> The rest of that log is either warnings about deprecated features or 
>> lines like these (which doesn't look fatal):
>> 01-Jun-2017 05:57:35.306 WARNING [localhost-startStop-1] 
>> org.apache.solr.handler.component.SpellCheckComponent.inform No 
>> queryConverter defined, using default converter
>>
>> I made the mistake of turning on DEBUG logging. But after filtering out 
>> thousands of DEBUG messages I see:
>> 2018-10-09 13:04:25,728 INFO  org.dspace.usage.LoggerUsageEventListener @ 
>> anonymous:session_id=7D74E0A56FFA3493875B4B5436B875C6:ip_addr=127.0.0.1:view_item:handle=11122/6045
>> 2018-10-09 13:04:25,738 ERROR org.dspace.statistics.SolrLogger @ 
>> org.apache.commons.httpclient.ProtocolException: The server 
>> scholarworks.alaska.edu failed to respond with a valid HTTP response
>> org.apache.solr.client.solrj.SolrServerException: 
>> org.apache.commons.httpclient.ProtocolException: The server 
>> scholarworks.alaska.edu failed to respond with a valid HTTP response
>>
>> Perhaps Solr is trying to use HTTP while the server is configured for 
>> (mostly) HTTPS with a few exceptions. There was something a co-worker had 
>> added to the web.xml file
>> to force HTTPS but I don't see it now. He might've removed it and put all 
>> that in the httpd configuration. If he didn't that might be the problem - 
>> the two of them fighting.
>>
>>
>> On Tuesday, October 9, 2018 at 11:51:14 AM UTC-8, Tim Donohue wrote:
>>>
>>> Hi Walter,
>>>
>>> I don't think those "messages.xml" notices have anything to do with this 
>>> Solr Statistics problem.  That "messages.xml" file is not directly related 
>>> to Solr.  The XMLUI also looks for those files in several different 
>>> locations, so if it doesn't find them in one place, it will look 
>>> elsewhere.  So, those log lines can likely be ignored.
>>>
>>> What you should be looking more closely at is whether any 
>>> errors/warnings are occurring in the dspace.log files (or Tomcat logs) when 
>>> you:
>>> (1) Access any Community/Collection/Item homepage on the site --- this 
>>> should contact Solr and log a "view" of that Item
>>> (2) When you access the statistics page (that empty page may be throwing 
>>> an error behind the scenes).
>>>
>>> Essentially look closely at the logs whenever you are performing tasks 
>>> that should be logging "view" statistics to Solr, and also look closely in 
>>> the Tomcat logs to be sure that the "/solr" webapp is deploying 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

I am occasionally seeing errors like this in the catalina.out file:

Error using query type: 2
[WARN] deprecation - The 'component-configurations' section in the sitemap 
is deprecated. Please check for alternatives.
Error using query *:*
Error using query statistics_type:workflow AND NOT(previousWorkflowStep: 
SUBMIT)
Error using query type: 2 AND  id:2695
Error using query type: 0 AND owningItem:2695
Error using query type: 2 AND  id:3677
Error using query type: 0 AND owningItem:3677

The four digit numbers appear to be valid item numbers so I'm not sure 
what's generating the errors. So I'm slogging
through the manual and sitemaps to see if I can trace what happens when the 
"View Usage Statistics" link is clicked.



On Tuesday, October 9, 2018 at 2:48:54 PM UTC-8, wlruth...@alaska.edu wrote:
>
> Rats... couldn't be that easy.
>
> Finding very few non-INFO messages in the logs. I did find an entry in 
> catalina.out that at least tells me it is/was running solr (perhaps with 
> memory leaks).
> 01-Jun-2017 01:23:38.504 WARNING [localhost-startStop-2] 
> org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The 
> web application [solr] appears to have started a thread named 
> [MultiThreadedHttpConnectionManager cleanu
> p] but has failed to stop it. This is very likely to create a memory leak. 
> Stack trace of thread:
>
> The rest of that log is either warnings about deprecated features or lines 
> like these (which doesn't look fatal):
> 01-Jun-2017 05:57:35.306 WARNING [localhost-startStop-1] 
> org.apache.solr.handler.component.SpellCheckComponent.inform No 
> queryConverter defined, using default converter
>
> I made the mistake of turning on DEBUG logging. But after filtering out 
> thousands of DEBUG messages I see:
> 2018-10-09 13:04:25,728 INFO  org.dspace.usage.LoggerUsageEventListener @ 
> anonymous:session_id=7D74E0A56FFA3493875B4B5436B875C6:ip_addr=127.0.0.1:view_item:handle=11122/6045
> 2018-10-09 13:04:25,738 ERROR org.dspace.statistics.SolrLogger @ 
> org.apache.commons.httpclient.ProtocolException: The server 
> scholarworks.alaska.edu failed to respond with a valid HTTP response
> org.apache.solr.client.solrj.SolrServerException: 
> org.apache.commons.httpclient.ProtocolException: The server 
> scholarworks.alaska.edu failed to respond with a valid HTTP response
>
> Perhaps Solr is trying to use HTTP while the server is configured for 
> (mostly) HTTPS with a few exceptions. There was something a co-worker had 
> added to the web.xml file
> to force HTTPS but I don't see it now. He might've removed it and put all 
> that in the httpd configuration. If he didn't that might be the problem - 
> the two of them fighting.
>
>
> On Tuesday, October 9, 2018 at 11:51:14 AM UTC-8, Tim Donohue wrote:
>>
>> Hi Walter,
>>
>> I don't think those "messages.xml" notices have anything to do with this 
>> Solr Statistics problem.  That "messages.xml" file is not directly related 
>> to Solr.  The XMLUI also looks for those files in several different 
>> locations, so if it doesn't find them in one place, it will look 
>> elsewhere.  So, those log lines can likely be ignored.
>>
>> What you should be looking more closely at is whether any errors/warnings 
>> are occurring in the dspace.log files (or Tomcat logs) when you:
>> (1) Access any Community/Collection/Item homepage on the site --- this 
>> should contact Solr and log a "view" of that Item
>> (2) When you access the statistics page (that empty page may be throwing 
>> an error behind the scenes).
>>
>> Essentially look closely at the logs whenever you are performing tasks 
>> that should be logging "view" statistics to Solr, and also look closely in 
>> the Tomcat logs to be sure that the "/solr" webapp is deploying properly, 
>> without errors.
>>
>> - Tim
>>
>> On Tue, Oct 9, 2018 at 2:41 PM  wrote:
>>
>>>
>>> This might be the problem. In the latest cocoon log I saw lots of 
>>> entries flash by that started with date/time like this:
>>> 2018-10-09 11:20:37,862 INFO  org.apache.cocoon.i18n.XMLResourceBundle 
>>> Then:
>>> - Bundle  
>>> not loaded: Source URI not found
>>> - Bundle  not loaded: 
>>> Source URI not found
>>> - Bundle 
>>>  not 
>>> loaded: Source URI not found
>>> - Bundle  not 
>>> loaded: Source URI not found
>>> - Bundle 
>>>  not 
>>> loaded: Source URI not found
>>> - Bundle  not 
>>> loaded: Source URI not found
>>> - Bundle 
>>>  not 
>>> loaded: Source URI not found
>>> - Bundle  not 
>>> loaded: Source URI not found
>>> - Bundle 
>>>  not 
>>> loaded: Source URI not found
>>> - Bundle  not 
>>> loaded: Source URI not found
>>> - Bundle 
>>>  not 
>>> loaded: Source URI not found
>>> - Bundle  not loaded: 
>>> Source URI not found
>>>
>>> The 'aspects' are something I've seen in the manual that's been added 
>>> with the upgrade but I've done nothing to
>>> configure anything there.
>>>
>>> I started to say the messages.xml file exists but no, the one I've had 
>>> to edit is in the i18n directory 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

This might be the problem. In the latest cocoon log I saw lots of entries 
flash by that started with date/time like this:
2018-10-09 11:20:37,862 INFO  org.apache.cocoon.i18n.XMLResourceBundle 
Then:
- Bundle  
not loaded: Source URI not found
- Bundle  not loaded: Source 
URI not found
- Bundle 
 not 
loaded: Source URI not found
- Bundle  not loaded: 
Source URI not found
- Bundle 
 not 
loaded: Source URI not found
- Bundle  not loaded: 
Source URI not found
- Bundle 
 not 
loaded: Source URI not found
- Bundle  not loaded: 
Source URI not found
- Bundle 
 not 
loaded: Source URI not found
- Bundle  not loaded: 
Source URI not found
- Bundle  
not loaded: Source URI not found
- Bundle  not loaded: 
Source URI not found

The 'aspects' are something I've seen in the manual that's been added with 
the upgrade but I've done nothing to
configure anything there.

I started to say the messages.xml file exists but no, the one I've had to 
edit is in the i18n directory - the one in the
error message doesn't exist in i18n/aspects/Versioning -  the entire 
'aspects' subdirectory doesn't exist at all.
I could create the directories but can't populate them without knowing what 
should be in there. Where should it
have come from? If fixing this doesn't require a scratch rebuild we might 
be able to quickly squash this bug.

   Walter R.


On Tuesday, October 9, 2018 at 11:12:43 AM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> All I meant is that the "solr" webapp that DSpace creates needs to be 
> available/deployed to Tomcat.  So, for example, if Tomcat is running on 
> http://localhost:8080, then Solr should be accessible on something like 
> http://localhost:8080/solr/
>
> In other words, the only way DSpace can properly communicate with Solr (to 
> send/query statistics) is if it's available in Tomcat.  The full path to 
> Solr in Tomcat should also be configured in your installed 
> [dspace]/config/modules/solr-statistics.cfg: 
> https://github.com/DSpace/DSpace/blob/dspace-5_x/dspace/config/modules/solr-statistics.cfg#L12
>
> If DSpace is unable to find/communicate with Solr, you likely would see 
> errors appearing in your logs.
>
> - Tim
>
> On Tue, Oct 9, 2018 at 2:04 PM > wrote:
>
>> Thank you Tim. I didn't notice anything in the logs that seemed to point 
>> to a problem with Solr
>> or the statistics. I'll look again to make sure I haven't overlooked 
>> something. Also checking the
>> troubleshooting page you linked.
>>
>> I'm not sure what you mean by the DSpace Solr webapp, *must* be installed 
>> into Tomcat. Are
>> you talking about during the build it must be configured with Tomcat or 
>> that it is configured in
>> the [dspace]/webapps/solr directory after DSpace has been built?
>>
>>Walter R.
>>
>>
>> On Tuesday, October 9, 2018 at 10:28:53 AM UTC-8, Tim Donohue wrote:
>>
>>> Hi Walter,
>>>
>>> I'd recommend checking your logs for any errors or warnings.  We have a 
>>> guide for doing so at:
>>> https://wiki.duraspace.org/display/DSPACE/Troubleshoot+an+error
>>>
>>> DSpace does *require* using Solr for Statistics.  However, the required 
>>> "solr" webapp is provided out-of-the-box with DSpace (it is built into the 
>>> [dspace]/webapps/solr/ directory), so you do not need to install Apache 
>>> Solr separately.  That DSpace Solr webapp *must* be installed into Tomcat 
>>> (alongside your UI, either XMLUI or JSPUI) for Statistics to work properly.
>>>
>>> Any errors that are occurring with accessing or logging statistics 
>>> should be logged in your dspace.log file (or, occasionally, in the Tomcat 
>>> logs).
>>>
>>> In the XMLUI, blank pages usually mean something went wrong.  Most of 
>>> the time, you should have more information in your logs.  However, if no 
>>> information is found there, it's always possible that the problem is in 
>>> your *theme* (XSLTs or similar).  So, if you get stuck, you may also want 
>>> to try temporarily switching to a default DSpace theme, to ensure your 
>>> custom theme is not causing issues.
>>>
>>> If you have more information to share, or start to figure out what might 
>>> be to blame, please let us know on this mailing list.  Currently, though, 
>>> it doesn't look like we have enough information to provide help (unless 
>>> someone else on this mailing list has seen this exact same behavior before).
>>>
>>> Tim
>>>
>>> On Mon, Oct 8, 2018 at 5:12 PM  wrote:
>>>
>>
 I'm still having problems with the statistics in DSpace. I have an 
 older server running DSpace3.1 which is
 a "copy" of the original system. The statistics apparently haven't 
 worked since the system moved. The 2nd
 system is running DSpace5.5. On both systems there is a "View Usage 
 Statistics" link on the Home page
 but following the link gives a valid, live page but with no numbers. 
 The Search and Workflow statistic links
 give identical error messages on both systems.

 I have gone through the manual and tried enabling all of the 

Re: [dspace-tech] View Statistics links not working

[wlrutherford]

Thank you Tim. I didn't notice anything in the logs that seemed to point to 
a problem with Solr
or the statistics. I'll look again to make sure I haven't overlooked 
something. Also checking the
troubleshooting page you linked.

I'm not sure what you mean by the DSpace Solr webapp, *must* be installed 
into Tomcat. Are
you talking about during the build it must be configured with Tomcat or 
that it is configured in
the [dspace]/webapps/solr directory after DSpace has been built?

   Walter R.


On Tuesday, October 9, 2018 at 10:28:53 AM UTC-8, Tim Donohue wrote:
>
> Hi Walter,
>
> I'd recommend checking your logs for any errors or warnings.  We have a 
> guide for doing so at:
> https://wiki.duraspace.org/display/DSPACE/Troubleshoot+an+error
>
> DSpace does *require* using Solr for Statistics.  However, the required 
> "solr" webapp is provided out-of-the-box with DSpace (it is built into the 
> [dspace]/webapps/solr/ directory), so you do not need to install Apache 
> Solr separately.  That DSpace Solr webapp *must* be installed into Tomcat 
> (alongside your UI, either XMLUI or JSPUI) for Statistics to work properly.
>
> Any errors that are occurring with accessing or logging statistics should 
> be logged in your dspace.log file (or, occasionally, in the Tomcat logs).
>
> In the XMLUI, blank pages usually mean something went wrong.  Most of the 
> time, you should have more information in your logs.  However, if no 
> information is found there, it's always possible that the problem is in 
> your *theme* (XSLTs or similar).  So, if you get stuck, you may also want 
> to try temporarily switching to a default DSpace theme, to ensure your 
> custom theme is not causing issues.
>
> If you have more information to share, or start to figure out what might 
> be to blame, please let us know on this mailing list.  Currently, though, 
> it doesn't look like we have enough information to provide help (unless 
> someone else on this mailing list has seen this exact same behavior before).
>
> Tim
>
> On Mon, Oct 8, 2018 at 5:12 PM > wrote:
>
>>
>> I'm still having problems with the statistics in DSpace. I have an older 
>> server running DSpace3.1 which is
>> a "copy" of the original system. The statistics apparently haven't worked 
>> since the system moved. The 2nd
>> system is running DSpace5.5. On both systems there is a "View Usage 
>> Statistics" link on the Home page
>> but following the link gives a valid, live page but with no numbers. The 
>> Search and Workflow statistic links
>> give identical error messages on both systems.
>>
>> I have gone through the manual and tried enabling all of the 
>> configuration parameters mentioned but with
>> no luck. I even tried a few which didn't at first seem necessary but 
>> couldn't hurt. Still nothing.
>>
>> The manual says that DSpace uses Apache Solr and there is no need to 
>> download any separate software.
>> But something isn't working. Do I really need to install and run Apache 
>> Solr? What I'm I misconfiguring?
>>
>> Does this look familiar to anyone?
>>
>> DSpace3.1 system: https://scholarworks.alaska.edu/statistics-home  
>> (may be behind firewall)
>> DSpace5.5 system: 
>> https://dspace36a.library.uaf.edu/xmlui/statistics-home
>>
>> Thank you,
>>
>>Walter Rutherford
>>
>> -- 
>> All messages to this mailing list should adhere to the DuraSpace Code of 
>> Conduct: https://duraspace.org/about/policies/code-of-conduct/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "DSpace Technical Support" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to dspace-tech...@googlegroups.com .
>> To post to this group, send email to dspac...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/dspace-tech.
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> Tim Donohue
> Technical Lead for DSpace & DSpaceDirect
> DuraSpace.org | DSpace.org | DSpaceDirect.org
>

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] View Statistics links not working

[wlrutherford]

I'm still having problems with the statistics in DSpace. I have an older 
server running DSpace3.1 which is
a "copy" of the original system. The statistics apparently haven't worked 
since the system moved. The 2nd
system is running DSpace5.5. On both systems there is a "View Usage 
Statistics" link on the Home page
but following the link gives a valid, live page but with no numbers. The 
Search and Workflow statistic links
give identical error messages on both systems.

I have gone through the manual and tried enabling all of the configuration 
parameters mentioned but with
no luck. I even tried a few which didn't at first seem necessary but 
couldn't hurt. Still nothing.

The manual says that DSpace uses Apache Solr and there is no need to 
download any separate software.
But something isn't working. Do I really need to install and run Apache 
Solr? What I'm I misconfiguring?

Does this look familiar to anyone?

DSpace3.1 system: https://scholarworks.alaska.edu/statistics-home  (may 
be behind firewall)
DSpace5.5 system: 
https://dspace36a.library.uaf.edu/xmlui/statistics-home

Thank you,

   Walter Rutherford

-- 
All messages to this mailing list should adhere to the DuraSpace Code of 
Conduct: https://duraspace.org/about/policies/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

Re: [dspace-tech] Re: Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

Reindex is complete. Browse counts are lower on new system.

I was hoping it was just a new way of indexing but I found an author who 
shows up on the old site but not the new
one. If I copy/paste the file handle to the new system I'm directed to the 
file so the files are on the new system, and
even in the right place, the system just doesn't where/how to find them 
unless told explicitly.

So now I'm poking through the webapps hierarchy (and PostgreSQL tables) to 
find where browse and search
specifications get converted into file handles.



On Wednesday, March 28, 2018 at 12:52:03 PM UTC-8, helix84 wrote:
>
> The Access Rights Awareness feature was added around DSpace 3. Usually 
> what people upgrading from an older version are surprised to see is DSpace 
> showing lower item counts than before for users who are not logged in.
>
>
> https://wiki.duraspace.org/display/DSDOC6x/Discovery#Discovery-AccessRightsAwareness
>
> https://jira.duraspace.org/browse/DS-1229
>
> Assuming that the reindex is complete (which you'd see running in 
> dspace.log), compare item counts in Browse by Title or Browse by Issue 
> Date. Then log in as administrator and compare them again, they should be 
> equal.
>
>
> Regards,
> ~~helix84
>
> Compulsory reading: DSpace Mailing List Etiquette
> https://wiki.duraspace.org/display/DSPACE/Mailing+List+Etiquette
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

Re: [dspace-tech] Re: Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

Curiouser and curiouser...

After one of my changes introduced weird formatting errors to the webpage I 
rolled back an earlier version.
On inspection I saw it was reporting even more records than before 
(differences of 123, 70, 0 from what I
was expecting). This was great news.
But an hour later when I checked again the differences had increased to the 
figures I noted before! What
happened? None of the dspace cron scripts were due to run for >12hrs so I 
doubt they were involved.

I've hunted for records which are missing on the new system to see if 
there's an obvious reason why they
were dropped. We did find a difference in the number of fields in one of 
the tables but I don't know if that's
even related to this issue.

What exactly is the difference between the old database format and the new? 
We haven't introduced any
modifications that I'm aware of so there must be another reason migration 
is partially failing for us.

.
On Tuesday, March 13, 2018 at 5:09:51 PM UTC-8, wlruth...@alaska.edu wrote:
>
> Yes, I checked browse by author, title, and subject. All were off a bit: 
> by 288, 182, and 11 records respectively. But I hadn't
> applied custom themes and *perhaps* some of the configuration changes. 
> I'm sure the customized themes can wait but I'll
> double-check that all of the configuration settings are being set 
> correctly. I also hadn't noticed any errors in the logs but I
> did skim them pretty quickly to verify I was seeing (thousands of) 
> re-ingest notices. I'll go through the logs with a finer comb.
>
> Thanks for the possible gotchas.
>
>
> On Tuesday, March 13, 2018 at 10:24:06 AM UTC-8, Tim Donohue wrote:
>>
>> Hi,
>>
>> Have you checked your DSpace logs to see if there were any errors 
>> reported there, either from the 'dspace database migrate', or from the 
>> full site reindex (over the weekend)?  That'd be the first place I'd 
>> recommend checking if there are missing items.
>>
>> More on checking logs for errors at: 
>> https://wiki.duraspace.org/display/DSPACE/Troubleshoot+an+error
>>
>> Also, I'd recommend comparing the number of items listed in "Browse by 
>> Title" between the two versions (DSpace 3.6 and 5.5).  That will give you 
>> better information on whether all the Items have been updated/upgraded 
>> successfully. It sounds like you are comparing the numbers in "Browse by 
>> Author"...and while those should pretty much match up, it is possible that 
>> DSpace 5 has reindexed your author information in such a way that it 
>> removed some duplicates that may have existed in DSpace 3.  
>>
>> You also may wish to verify that any configurations from your old DSpace 
>> (version 3.6) have been brought forward into DSpace 5.5. So, for example, 
>> if you customized Browse by Author settings (in dspace.cfg) to also use 
>> extra fields (not just "dc.contributor.author"), then you should verify 
>> that DSpace 5 is using those same fields (if it is not, then its author 
>> count would be lower).
>>
>> That's just a couple of ideas that come to mind. Hopefully one of them 
>> will help you start to figure out what may be going on.  As always, let us 
>> know if you have further questions or figure out more of what may be 
>> happening.
>>
>> Good luck,
>>
>> Tim
>>
>> On Tue, Mar 13, 2018 at 12:37 PM  wrote:
>>
>>> I gave it the weekend to be sure it was done then manually ran 'dspace 
>>> database migrate' again. If returned much quicker but nothing
>>> had changed. Of the 5105 author records I expected it found 4817. I 
>>> guess that's not bad. If I knew a way to tell what it's not finding
>>> it might not be too much work to just reentering them by hand. But so 
>>> far I haven't turned up what is different between the two copies.
>>>
>>>
>>> On Thursday, March 8, 2018 at 9:19:00 AM UTC-9, Tim Donohue wrote:
>>>
 Hello,

 While I'm not certain this is the case, it *sounds* like the behavior 
 you are seeing is related to a full reindex of the site.

 After any database upgrade (via `dspace database migrate`), DSpace will 
 trigger a full reindex of your site the next time you start up Tomcat (or 
 whatever servlet container you are using). This full reindex may take some 
 time (minutes or even hours, depending on the amount of content), but it 
 runs in the backend. A reindex is required for major upgrades, as often 
 the 
 database structure changes, and your search/browse indexes then need to be 
 updated based on those changes.

 You may wish to check your DSpace log files to see if you are seeing 
 messages related to individual Items being "indexed".  If so, this is the 
 reindex processing each file, and once it is complete all your records 
 should reappear in the search/browse.  While it is reindexing you still 
 can 
 use the site, but not all records may appear (they each will appear as 
 they 
 are reindexed).

 And just to clarify, 

Re: [dspace-tech] Re: Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

Yes, I checked browse by author, title, and subject. All were off a bit: by 
288, 182, and 11 records respectively. But I hadn't
applied custom themes and *perhaps* some of the configuration changes. I'm 
sure the customized themes can wait but I'll
double-check that all of the configuration settings are being set 
correctly. I also hadn't noticed any errors in the logs but I
did skim them pretty quickly to verify I was seeing (thousands of) 
re-ingest notices. I'll go through the logs with a finer comb.

Thanks for the possible gotchas.


On Tuesday, March 13, 2018 at 10:24:06 AM UTC-8, Tim Donohue wrote:
>
> Hi,
>
> Have you checked your DSpace logs to see if there were any errors reported 
> there, either from the 'dspace database migrate', or from the full site 
> reindex (over the weekend)?  That'd be the first place I'd recommend 
> checking if there are missing items.
>
> More on checking logs for errors at: 
> https://wiki.duraspace.org/display/DSPACE/Troubleshoot+an+error
>
> Also, I'd recommend comparing the number of items listed in "Browse by 
> Title" between the two versions (DSpace 3.6 and 5.5).  That will give you 
> better information on whether all the Items have been updated/upgraded 
> successfully. It sounds like you are comparing the numbers in "Browse by 
> Author"...and while those should pretty much match up, it is possible that 
> DSpace 5 has reindexed your author information in such a way that it 
> removed some duplicates that may have existed in DSpace 3.  
>
> You also may wish to verify that any configurations from your old DSpace 
> (version 3.6) have been brought forward into DSpace 5.5. So, for example, 
> if you customized Browse by Author settings (in dspace.cfg) to also use 
> extra fields (not just "dc.contributor.author"), then you should verify 
> that DSpace 5 is using those same fields (if it is not, then its author 
> count would be lower).
>
> That's just a couple of ideas that come to mind. Hopefully one of them 
> will help you start to figure out what may be going on.  As always, let us 
> know if you have further questions or figure out more of what may be 
> happening.
>
> Good luck,
>
> Tim
>
> On Tue, Mar 13, 2018 at 12:37 PM  
> wrote:
>
>> I gave it the weekend to be sure it was done then manually ran 'dspace 
>> database migrate' again. If returned much quicker but nothing
>> had changed. Of the 5105 author records I expected it found 4817. I guess 
>> that's not bad. If I knew a way to tell what it's not finding
>> it might not be too much work to just reentering them by hand. But so far 
>> I haven't turned up what is different between the two copies.
>>
>>
>> On Thursday, March 8, 2018 at 9:19:00 AM UTC-9, Tim Donohue wrote:
>>
>>> Hello,
>>>
>>> While I'm not certain this is the case, it *sounds* like the behavior 
>>> you are seeing is related to a full reindex of the site.
>>>
>>> After any database upgrade (via `dspace database migrate`), DSpace will 
>>> trigger a full reindex of your site the next time you start up Tomcat (or 
>>> whatever servlet container you are using). This full reindex may take some 
>>> time (minutes or even hours, depending on the amount of content), but it 
>>> runs in the backend. A reindex is required for major upgrades, as often the 
>>> database structure changes, and your search/browse indexes then need to be 
>>> updated based on those changes.
>>>
>>> You may wish to check your DSpace log files to see if you are seeing 
>>> messages related to individual Items being "indexed".  If so, this is the 
>>> reindex processing each file, and once it is complete all your records 
>>> should reappear in the search/browse.  While it is reindexing you still can 
>>> use the site, but not all records may appear (they each will appear as they 
>>> are reindexed).
>>>
>>> And just to clarify, you no longer need to upgrade from one major 
>>> version to the next.  So, you should not need to upgrade from DSpace 3 to 5 
>>> to 6.  You can now *directly* upgrade from any prior release to DSpace 6 
>>> (even from DSpace 1.0 -> 6.0 would work).  The only requirement is that you 
>>> need to make sure your database backend is upgraded first, to a compatible 
>>> release.  So, for example, if you are running a very old version of 
>>> Postgres or Oracle, you'd want to upgrade that database backend to a recent 
>>> version (as required by DSpace 6), and then you should be able to upgrade 
>>> directly to DSpace 6 per the upgrade instructions at: 
>>> https://wiki.duraspace.org/display/DSDOC6x/Upgrading+DSpace
>>>
>>> Hopefully that helps! If you have further questions, feel free to ask 
>>> them on this list.
>>>
>>> Tim
>>>
>>> On Thu, Mar 8, 2018 at 12:04 PM  wrote:
>>>
>> Interesting... It looks like the number of records has updated itself 
 since yesterday.
 Perhaps the automatic database migration is working, just slowly. I'll 
 watch it.


 On Wednesday, 

Re: [dspace-tech] Re: Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

I gave it the weekend to be sure it was done then manually ran 'dspace 
database migrate' again. If returned much quicker but nothing
had changed. Of the 5105 author records I expected it found 4817. I guess 
that's not bad. If I knew a way to tell what it's not finding
it might not be too much work to just reentering them by hand. But so far I 
haven't turned up what is different between the two copies.

On Thursday, March 8, 2018 at 9:19:00 AM UTC-9, Tim Donohue wrote:
>
> Hello,
>
> While I'm not certain this is the case, it *sounds* like the behavior you 
> are seeing is related to a full reindex of the site.
>
> After any database upgrade (via `dspace database migrate`), DSpace will 
> trigger a full reindex of your site the next time you start up Tomcat (or 
> whatever servlet container you are using). This full reindex may take some 
> time (minutes or even hours, depending on the amount of content), but it 
> runs in the backend. A reindex is required for major upgrades, as often the 
> database structure changes, and your search/browse indexes then need to be 
> updated based on those changes.
>
> You may wish to check your DSpace log files to see if you are seeing 
> messages related to individual Items being "indexed".  If so, this is the 
> reindex processing each file, and once it is complete all your records 
> should reappear in the search/browse.  While it is reindexing you still can 
> use the site, but not all records may appear (they each will appear as they 
> are reindexed).
>
> And just to clarify, you no longer need to upgrade from one major version 
> to the next.  So, you should not need to upgrade from DSpace 3 to 5 to 6.  
> You can now *directly* upgrade from any prior release to DSpace 6 (even 
> from DSpace 1.0 -> 6.0 would work).  The only requirement is that you need 
> to make sure your database backend is upgraded first, to a compatible 
> release.  So, for example, if you are running a very old version of 
> Postgres or Oracle, you'd want to upgrade that database backend to a recent 
> version (as required by DSpace 6), and then you should be able to upgrade 
> directly to DSpace 6 per the upgrade instructions at: 
> https://wiki.duraspace.org/display/DSDOC6x/Upgrading+DSpace
>
> Hopefully that helps! If you have further questions, feel free to ask them 
> on this list.
>
> Tim
>
> On Thu, Mar 8, 2018 at 12:04 PM  
> wrote:
>
>> Interesting... It looks like the number of records has updated itself 
>> since yesterday.
>> Perhaps the automatic database migration is working, just slowly. I'll 
>> watch it.
>>
>>
>> On Wednesday, March 7, 2018 at 3:17:03 PM UTC-9, wlruth...@alaska.edu 
>> wrote:
>>>
>>>
>>> While upgrading our repository we initially built DSpace-6.0 but it was 
>>> unable to fully ingest
>>> the legacy postgres database used by the older DSpace-3.1 system. We 
>>> were told we'd need
>>> to first install DSpace-3.x, upgrade that to DSpace-5.x, then upgrade to 
>>> DSpace-6.0.
>>>
>>> So we built a DSpace-3.6 system and it was able to read the DSpace-3.1 
>>> database. We then
>>> built a DSpace-5.5 system with a copy of the entire database. All seemed 
>>> well at first but on
>>> closer inspection it's doing the same thing as our original DSpace-6.0 
>>> system - ignoring most
>>> of the database. The manual says DSpace will automatically fix the 
>>> database or that the
>>> 'dspace database migrate' command will accomplish this. Neither seems to 
>>> have had any
>>> effect. I'm scanning through logfiles to see if it silently told me that 
>>> something went wrong.
>>>
>>> I know other DSpace users have successfully upgraded from DSpace-3.x to 
>>> DSpace-6.0 so
>>> that's encouraging. Am I missing a step? Do I have install DSpace-4.x 
>>> before DSpace-5.x?
>>>
>>> Thank you for any help you can provide.
>>>
>>>
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "DSpace Technical Support" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to dspace-tech...@googlegroups.com .
>> To post to this group, send email to dspac...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/dspace-tech.
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> Tim Donohue
> Technical Lead for DSpace & DSpaceDirect
> DuraSpace.org | DSpace.org | DSpaceDirect.org
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Re: Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

Interesting... It looks like the number of records has updated itself since 
yesterday.
Perhaps the automatic database migration is working, just slowly. I'll 
watch it.


On Wednesday, March 7, 2018 at 3:17:03 PM UTC-9, wlruth...@alaska.edu wrote:
>
>
> While upgrading our repository we initially built DSpace-6.0 but it was 
> unable to fully ingest
> the legacy postgres database used by the older DSpace-3.1 system. We were 
> told we'd need
> to first install DSpace-3.x, upgrade that to DSpace-5.x, then upgrade to 
> DSpace-6.0.
>
> So we built a DSpace-3.6 system and it was able to read the DSpace-3.1 
> database. We then
> built a DSpace-5.5 system with a copy of the entire database. All seemed 
> well at first but on
> closer inspection it's doing the same thing as our original DSpace-6.0 
> system - ignoring most
> of the database. The manual says DSpace will automatically fix the 
> database or that the
> 'dspace database migrate' command will accomplish this. Neither seems to 
> have had any
> effect. I'm scanning through logfiles to see if it silently told me that 
> something went wrong.
>
> I know other DSpace users have successfully upgraded from DSpace-3.x to 
> DSpace-6.0 so
> that's encouraging. Am I missing a step? Do I have install DSpace-4.x 
> before DSpace-5.x?
>
> Thank you for any help you can provide.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Database migration DSpace-3.6 to DSpace-5.5

[wlrutherford]

While upgrading our repository we initially built DSpace-6.0 but it was 
unable to fully ingest
the legacy postgres database used by the older DSpace-3.1 system. We were 
told we'd need
to first install DSpace-3.x, upgrade that to DSpace-5.x, then upgrade to 
DSpace-6.0.

So we built a DSpace-3.6 system and it was able to read the DSpace-3.1 
database. We then
built a DSpace-5.5 system with a copy of the entire database. All seemed 
well at first but on
closer inspection it's doing the same thing as our original DSpace-6.0 
system - ignoring most
of the database. The manual says DSpace will automatically fix the database 
or that the
'dspace database migrate' command will accomplish this. Neither seems to 
have had any
effect. I'm scanning through logfiles to see if it silently told me that 
something went wrong.

I know other DSpace users have successfully upgraded from DSpace-3.x to 
DSpace-6.0 so
that's encouraging. Am I missing a step? Do I have install DSpace-4.x 
before DSpace-5.x?

Thank you for any help you can provide.



-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Re: Statistics: Invalid content type

[wlrutherford]

Hello Jeffrey,

Did you ever find a solution? I'm seeing the same errors. The difference is 
I'm not sure if the Usage Statistics
ever worked. I recently built a new server and moved the database to 
replace an older system managed by
somebody else. Users are reporting that the stats worked on the old server 
but not now.



On Wednesday, May 24, 2017 at 10:49:45 AM UTC-8, Jeffrey Sheldon wrote:
>
> Folks, 
>
> Our usage statistics abruptly stopped working yesterday despite no major 
> changes having taken place in our DSpace 5.3/XMLUI installation.  If I 
> choose to view statistics, nothing is displayed.  Rather, I receive the 
> following errors in catalina.out after clicking on the associated action: 
>
> View Usage Statistics 
> Error using query type: 2 
>
> View Search Statistics 
> Error using query *:* 
>
> View Workflow Statistics 
> Error using query statistics_type:workflow AND NOT(previousWorkflowStep: 
> SUBMIT) 
>
> In the dspace.log file, this is the response for "View Usage Statistics": 
>
> 2017-05-24 18:31:47,344 ERROR 
> org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer @ Error 
> occurred while creating statistics for home page 
> org.apache.solr.client.solrj.SolrServerException: Error executing query 
> at 
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:100)
>  
>
> at 
> org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301) 
> at org.dspace.statistics.SolrLogger.query(SolrLogger.java:1203) 
> at 
> org.dspace.statistics.SolrLogger.queryFacetField(SolrLogger.java:920) 
> at 
> org.dspace.statistics.content.StatisticsDataVisits.queryFacetField(StatisticsDataVisits.java:647)
>  
>
> at 
> org.dspace.statistics.content.StatisticsDataVisits.createDataset(StatisticsDataVisits.java:252)
>  
>
> at 
> org.dspace.statistics.content.StatisticsDisplay.getDataset(StatisticsDisplay.java:88)
>  
>
> at 
> org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer.addDisplayListing(StatisticsTransformer.java:373)
>  
>
> at 
> org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer.renderHome(StatisticsTransformer.java:156)
>  
>
> at 
> org.dspace.app.xmlui.aspect.statistics.StatisticsTransformer.addBody(StatisticsTransformer.java:106)
>  
>
> at 
> org.dspace.app.xmlui.wing.AbstractWingTransformer.startElement(AbstractWingTransformer.java:223)
>  
>
> at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  
>
> at java.lang.reflect.Method.invoke(Method.java:606) 
> at 
> org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
>  
>
> at com.sun.proxy.$Proxy99.startElement(Unknown Source) 
> at 
> org.apache.cocoon.components.sax.XMLTeePipe.startElement(XMLTeePipe.java:87) 
>
> at 
> org.apache.cocoon.components.sax.AbstractXMLByteStreamInterpreter.parse(AbstractXMLByteStreamInterpreter.java:117)
>  
>
> at 
> org.apache.cocoon.components.sax.XMLByteStreamInterpreter.deserialize(XMLByteStreamInterpreter.java:44)
>  
>
> at 
> org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.processXMLPipeline(AbstractCachingProcessingPipeline.java:324)
>  
>
> at 
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:750)
>  
>
> at sun.reflect.GeneratedMethodAccessor197.invoke(Unknown Source) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  
>
> at java.lang.reflect.Method.invoke(Method.java:606) 
> at 
> org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
>  
>
> at com.sun.proxy.$Proxy87.process(Unknown Source) 
> at 
> org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:362)
>  
>
> at 
> org.apache.cocoon.components.source.util.SourceUtil.toSAX(SourceUtil.java:111)
>  
>
> at 
> org.apache.cocoon.components.source.util.SourceUtil.parse(SourceUtil.java:294)
>  
>
> at 
> org.apache.cocoon.generation.FileGenerator.generate(FileGenerator.java:136) 
> at sun.reflect.GeneratedMethodAccessor188.invoke(Unknown Source) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  
>
> at java.lang.reflect.Method.invoke(Method.java:606) 
> at 
> org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
>  
>
> at com.sun.proxy.$Proxy92.generate(Unknown Source) 
> at 
> org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.processXMLPipeline(AbstractProcessingPipeline.java:544)
>  
>
> at 
> 

[dspace-tech] Re: statistics.item.authorization.admin format and default

[wlrutherford]

I should have specified that we're currently running DSpace 3.6 but going 
to upgrade to
Dspace 6.0 once everything is stable.  WLR


On Thursday, December 7, 2017 at 10:30:42 AM UTC-9, wlruth...@alaska.edu 
wrote:
>
> Thank you very much! I was waiting for a quiet time to set it to a very 
> low number but, if the default is
> zero, then that obviously won't solve the issue.
>
>Walter R.
>
> On Wednesday, December 6, 2017 at 5:54:55 AM UTC-9, Mark H. Wood wrote:
>>
>> On Tuesday, December 5, 2017 at 7:11:54 PM UTC-5, wlruth...@alaska.edu 
>> wrote:
>>>
>>> In the dspace.cfg file if a setting with a value is commented out it 
>>> typically is showing the default value.
>>> We have a user who isn't seeing updates to the communities and 
>>> collections. I found this line in the
>>> dspace.cfg file which *might* be the culprit:
>>> #xmlui.community-list.cache = 12 hours
>>>
>>> I'm accustomed to times being specified in numeric seconds or sometimes 
>>> minutes. This is unusual.
>>>
>>> Does that mean that 12hrs is the default value used by DSpace if a new 
>>> value isn't specified? What if
>>> our site is relatively small with few changes. Can it be turned off 
>>> completely by setting it to zero (0)?
>>> And what is the format of the time? Could I set 
>>> xmlui.community-list.cache = 1 minute? 30 seconds?
>>>
>>> I've searched the manual and online but didn't find the answer to these 
>>> question. So, any insight you
>>> can provide is greatly appreciated.
>>>
>>>
>>
>> As one might suppose, the code is in dspace-xmlui.  Specifically it is in 
>> org.dspace.app.xmlui.aspect.artifactbrowser.CommunityBrowser.  The value of 
>> xmlui.community-list.cache is passed to 
>> org.dspace.app.xmlui.util.DSpaceValidity.setAssumedValidityDelay, where it 
>> is parsed:
>>  * This method takes a string which is parsed for the delay time, the 
>> string 
>>  * must be of the following form: " " where scale is 
>> days,
>>  * hours, minutes, or seconds.
>>  * 
>>  * Examples: "1 day" or "12 hours" or "1 hour" or "30 minutes"
>> The code does accept either singular or plural and doesn't care if this 
>> matches the number -- "1 days" or "33 minute" will be accepted.  The unit 
>> of measure is required.
>>
>> It appears that, if unconfigured, the cache validity period is zero:  the 
>> underlying objects will always be checked for changes.  The numeric value 
>> is not checked in any way, so long as it is indeed a Java long integer, so 
>> you could set it to zero if you wish, but I think that will have the same 
>> effect as leaving it unconfigured.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Re: statistics.item.authorization.admin format and default

[wlrutherford]

Thank you very much! I was waiting for a quiet time to set it to a very low 
number but, if the default is
zero, then that obviously won't solve the issue.

   Walter R.

On Wednesday, December 6, 2017 at 5:54:55 AM UTC-9, Mark H. Wood wrote:
>
> On Tuesday, December 5, 2017 at 7:11:54 PM UTC-5, wlruth...@alaska.edu 
>  wrote:
>>
>> In the dspace.cfg file if a setting with a value is commented out it 
>> typically is showing the default value.
>> We have a user who isn't seeing updates to the communities and 
>> collections. I found this line in the
>> dspace.cfg file which *might* be the culprit:
>> #xmlui.community-list.cache = 12 hours
>>
>> I'm accustomed to times being specified in numeric seconds or sometimes 
>> minutes. This is unusual.
>>
>> Does that mean that 12hrs is the default value used by DSpace if a new 
>> value isn't specified? What if
>> our site is relatively small with few changes. Can it be turned off 
>> completely by setting it to zero (0)?
>> And what is the format of the time? Could I set 
>> xmlui.community-list.cache = 1 minute? 30 seconds?
>>
>> I've searched the manual and online but didn't find the answer to these 
>> question. So, any insight you
>> can provide is greatly appreciated.
>>
>>
>
> As one might suppose, the code is in dspace-xmlui.  Specifically it is in 
> org.dspace.app.xmlui.aspect.artifactbrowser.CommunityBrowser.  The value of 
> xmlui.community-list.cache is passed to 
> org.dspace.app.xmlui.util.DSpaceValidity.setAssumedValidityDelay, where it 
> is parsed:
>  * This method takes a string which is parsed for the delay time, the 
> string 
>  * must be of the following form: " " where scale is 
> days,
>  * hours, minutes, or seconds.
>  * 
>  * Examples: "1 day" or "12 hours" or "1 hour" or "30 minutes"
> The code does accept either singular or plural and doesn't care if this 
> matches the number -- "1 days" or "33 minute" will be accepted.  The unit 
> of measure is required.
>
> It appears that, if unconfigured, the cache validity period is zero:  the 
> underlying objects will always be checked for changes.  The numeric value 
> is not checked in any way, so long as it is indeed a Java long integer, so 
> you could set it to zero if you wish, but I think that will have the same 
> effect as leaving it unconfigured.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] statistics.item.authorization.admin format and default

[wlrutherford]

Seasons greetings all,

In the dspace.cfg file if a setting with a value is commented out it 
typically is showing the default value.
We have a user who isn't seeing updates to the communities and collections. 
I found this line in the
dspace.cfg file which *might* be the culprit:
#xmlui.community-list.cache = 12 hours

I'm accustomed to times being specified in numeric seconds or sometimes 
minutes. This is unusual.

Does that mean that 12hrs is the default value used by DSpace if a new 
value isn't specified? What if
our site is relatively small with few changes. Can it be turned off 
completely by setting it to zero (0)?
And what is the format of the time? Could I set xmlui.community-list.cache 
= 1 minute? 30 seconds?

I've searched the manual and online but didn't find the answer to these 
question. So, any insight you
can provide is greatly appreciated.

Thank you,

Walter R.

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

[dspace-tech] Re: [Dspace-tech] System-wide Alerts deactivated with Tomcat restart

[wlrutherford]

I've noticed the same behavior and it surprised us - we weren't expecting a 
restart to also reset the warnings. It's a bit annoying when you're trying 
to fix a problem that requires restarting the server because it can allow 
users to access the system (before you replace the warning) while you're in 
the middle of troubleshooting. But I have a new wrinkle on that issue.

We noticed that the warning message sometimes showed up and other times it 
didn't. It appeared to be random until I noticed that if the URL contained 
"/xmlui" (our default) that the warning showed up but without it you got no 
warning message.
In our webapps directory I'd created a simple link from ROOT (the default 
for Tomcat) to xmlui with dspace.url set to /xmlui. Removing the 
ROOT link breaks simple "https:///". So I removed "/xmlui" from 
dspace.url and restarted. The behavior was the same. Even more annoying is 
that if I am logged in and switch from a "/xmlui" page to one without 
"/xmlui" the system seems to log me out! Again, completely unexpected.

Has anyone else seen these behaviors? I'll keep looking for a cause and a 
cure.


On Tuesday, August 25, 2015 at 1:16:05 PM UTC-8, Jim Coble wrote:
>
> DSpace 1.7.1 
> We have noticed that, if you activate a System-wide Alert (with no 
> countdown timer) from the administrative Control Panel and then stop and 
> restart Tomcat, when the XMLUI DSpace web app comes back up, the 
> System-wide Alert is no longer active.  We're curious whether this is 
> expected behavior or whether the Alert should remain activated even through 
> a Tomcat restart until we explicitly deactivate it.  We use the System-wide 
> Alert functionality to announce scheduled downtime, sometimes days in 
> advance, and it appears that, if there is an unscheduled restart in the 
> meantime, our announcement is deactivated.  Not a big deal, just curious. 
>  Thanks. 
> --Jim 
>
>  
> Jim Coble 
> Core Services Technical Lead and Program Coordinator 
> Information Technology Services 
> Perkins Library 
> Email: jim@duke.edu  
> Voice: 919-660-5974  Fax: 919-668-2578 
> Box 90196, Duke University 
> Durham, NC 27708-0196 
>  
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to dspace-tech+unsubscr...@googlegroups.com.
To post to this group, send email to dspace-tech@googlegroups.com.
Visit this group at https://groups.google.com/group/dspace-tech.
For more options, visit https://groups.google.com/d/optout.

Re: [dspace-tech] Tomcat Cert Process for Dspace

[wlrutherford]

I may have it fixed but to do it I had to ignore the handle setup 
instructions and use
port 443 instead of port 8000. That forces a secure reference. Nothing 
appears to
happen when I click on the URI. Since http://hdl.handle.net/11122/ 
points
right back to the same page containing the link that makes perfect sense.


On Wednesday, October 25, 2017 at 1:11:49 PM UTC-8, wlruth...@alaska.edu 
wrote:
>
>
>
> On Wednesday, October 25, 2017 at 8:56:57 AM UTC-8, Mark H. Wood wrote:
>>
>> On Tuesday, October 24, 2017 at 9:46:58 PM UTC-4, wlruth...@alaska.edu 
>> wrote:
>>>
>>> We are using port 443 for secure connections and securing the site by 
>>> rewriting
>>> port 80 to also go to port 443. The handle server uses ports 2641 and 
>>> 8000 and
>>> needs them to to be non-SSL. But something is converting our 
>>> http://hdl.handle.net/
>>> URI links to https. So, rather than linking to the document page, we get 
>>> something
>>> like this:
>>>
>>> 
>>> 
>>> 400 Bad Request
>>> 
>>> Bad Request
>>> Your browser sent a request that this server could not understand.
>>> Reason: You're speaking plain HTTP to an SSL-enabled server port.
>>> Instead use the HTTPS scheme to access this URL, please.
>>> Hint: https://scholarworks.alaska.edu/ 
>>> ">https://scholarworks.alaska.edu/
>>> 
>>> Apache/2.2.15 (Red Hat) Server at scholarworks.alaska.edu Port 
>>> 443
>>> 
>>>
>>>
>>>
>>
>> I tried following the URI: link at 
>> https://scholarworks.alaska.edu/xmlui/handle/11122/6433 using 'curl'.  
>> The Handle link there is 'http://hdl.handle.net/11122/6433'.  
>> hdl.handle.net responded with a 303 redirect to '
>> http://scholarworks.alaska.edu:443/xmlui/handle/11122/6433' which is 
>> wrong:  it should either use 'https:' or omit the port 443.  I suspect that 
>> you've set 'dspace.url' to 'http://scholarworks.alaska.edu:443' instead 
>> of 'https://scholarworks.alaska.edu:443'.  The DSpace Handle resolver 
>> just appends the object's handle to the value of 'dspace.url' and sends 
>> that back to the master resolver at hdl.net.
>>
>>  
>  You're right, there was a mismatch in the dspace.cfg file between http:// 
> and the secure port 443. Unfortunately, changing dspace.url to clean 
> https:// didn't fix the problem. Same error.
>
>  
>>
>>> My best guess at the moment is that the virtualhost configurations are
>>> also somehow catching other ports and rewriting them in the same way.
>>>
>>> 
>>> #
>>> ### Redirect insecure ports to secure port 443
>>> ServerAdmin uaf-libra...@alaska.edu
>>> RewriteEngine On
>>> RewriteCond %{HTTPS} off
>>> #   RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]
>>> RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI}
>>>
>>
>>
>> This probably doesn't accomplish anything, since a rewrite to the same 
>> host will be stripped down to the localpart.  Forcing a redirect, as in the 
>> line above it, should evoke a new request to the proper port.  Try 
>> appending '[R,L]' or '[R=permanent,L]'.
>>
>>  
>>
>> DocumentRoot /dspace/webapps/xmlui
>>> ServerName scholarworks.alaska.edu
>>> ServerAlias scholarworks
>>>
>>> ErrorLog logs/scholarworks.alaska.edu-error_log
>>> CustomLog logs/scholarworks.alaska.edu-access_log common
>>>
>>> ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
>>> ### ^ this tells httpd to redirect it's / to localhost port 443
>>>
>>>
>>
>> No.  This is not a redirect.  HTTPD will act like a client and ask the 
>> target for the localpart, then send the reply back to *its* client.  Above 
>> HTTPD is proxying through itself, which is probably not what you want.  
>> Here we have no proxy rules at all for port 80, only actual redirects 
>> telling HTTPD's client to try again on port 443.  All the proxying happens 
>> in the port 443 virtual host.  I would just take all this proxy stuff out 
>> of this vhost.
>>
>> I commented out the proxyPass lines for  without 
> breaking anything. I also added the "[R,L]" per your suggestion. So far 
> I've noticed no difference.
>  
>
>>  
>>
>>> #timeout=300
>>> ProxyPassReverse / http://127.0.0.1:443/  retry=10
>>> ### ^ this tells httpd that tomcat's url's should be rewritten 
>>> to look
>>> ###   like they're coming from httpd.
>>>
>>> ProxyPreserveHost On
>>> ### ^ this tells httpd to keep the Host: information from the 
>>> client and
>>> ### pass it on to tomcat.
>>>
>>> 
>>>
>>> #
>>> 
>>>
>>> ### ^ this creates a httpd server that listens on port 443.
>>>
>>> ServerAdmin uaf-libra...@alaska.edu
>>> DocumentRoot /dspace/webapps/xmlui
>>> ServerName scholarworks.alaska.edu
>>> ServerAlias scholarworks
>>>
>>> ErrorLog logs/scholarworks.alaska.edu-https-error_log
>>> CustomLog 

Re: [dspace-tech] Tomcat Cert Process for Dspace

[wlrutherford]

On Wednesday, October 25, 2017 at 8:56:57 AM UTC-8, Mark H. Wood wrote:
>
> On Tuesday, October 24, 2017 at 9:46:58 PM UTC-4, wlruth...@alaska.edu 
>  wrote:
>>
>> We are using port 443 for secure connections and securing the site by 
>> rewriting
>> port 80 to also go to port 443. The handle server uses ports 2641 and 
>> 8000 and
>> needs them to to be non-SSL. But something is converting our 
>> http://hdl.handle.net/
>> URI links to https. So, rather than linking to the document page, we get 
>> something
>> like this:
>>
>> 
>> 
>> 400 Bad Request
>> 
>> Bad Request
>> Your browser sent a request that this server could not understand.
>> Reason: You're speaking plain HTTP to an SSL-enabled server port.
>> Instead use the HTTPS scheme to access this URL, please.
>> Hint: https://scholarworks.alaska.edu/ 
>> ">https://scholarworks.alaska.edu/
>> 
>> Apache/2.2.15 (Red Hat) Server at scholarworks.alaska.edu Port 
>> 443
>> 
>>
>>
>>
>
> I tried following the URI: link at 
> https://scholarworks.alaska.edu/xmlui/handle/11122/6433 using 'curl'.  
> The Handle link there is 'http://hdl.handle.net/11122/6433'.  
> hdl.handle.net responded with a 303 redirect to '
> http://scholarworks.alaska.edu:443/xmlui/handle/11122/6433' which is 
> wrong:  it should either use 'https:' or omit the port 443.  I suspect that 
> you've set 'dspace.url' to 'http://scholarworks.alaska.edu:443' instead 
> of 'https://scholarworks.alaska.edu:443'.  The DSpace Handle resolver 
> just appends the object's handle to the value of 'dspace.url' and sends 
> that back to the master resolver at hdl.net.
>
>  
 You're right, there was a mismatch in the dspace.cfg file between http:// 
and the secure port 443. Unfortunately, changing dspace.url to clean 
https:// didn't fix the problem. Same error.

 
>
>> My best guess at the moment is that the virtualhost configurations are
>> also somehow catching other ports and rewriting them in the same way.
>>
>> 
>> #
>> ### Redirect insecure ports to secure port 443
>> ServerAdmin uaf-libra...@alaska.edu 
>> RewriteEngine On
>> RewriteCond %{HTTPS} off
>> #   RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]
>> RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI}
>>
>
>
> This probably doesn't accomplish anything, since a rewrite to the same 
> host will be stripped down to the localpart.  Forcing a redirect, as in the 
> line above it, should evoke a new request to the proper port.  Try 
> appending '[R,L]' or '[R=permanent,L]'.
>
>  
>
> DocumentRoot /dspace/webapps/xmlui
>> ServerName scholarworks.alaska.edu
>> ServerAlias scholarworks
>>
>> ErrorLog logs/scholarworks.alaska.edu-error_log
>> CustomLog logs/scholarworks.alaska.edu-access_log common
>>
>> ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
>> ### ^ this tells httpd to redirect it's / to localhost port 443
>>
>>
>
> No.  This is not a redirect.  HTTPD will act like a client and ask the 
> target for the localpart, then send the reply back to *its* client.  Above 
> HTTPD is proxying through itself, which is probably not what you want.  
> Here we have no proxy rules at all for port 80, only actual redirects 
> telling HTTPD's client to try again on port 443.  All the proxying happens 
> in the port 443 virtual host.  I would just take all this proxy stuff out 
> of this vhost.
>
> I commented out the proxyPass lines for  without 
breaking anything. I also added the "[R,L]" per your suggestion. So far 
I've noticed no difference.
 

>  
>
>> #timeout=300
>> ProxyPassReverse / http://127.0.0.1:443/  retry=10
>> ### ^ this tells httpd that tomcat's url's should be rewritten to 
>> look
>> ###   like they're coming from httpd.
>>
>> ProxyPreserveHost On
>> ### ^ this tells httpd to keep the Host: information from the 
>> client and
>> ### pass it on to tomcat.
>>
>> 
>>
>> #
>> 
>>
>> ### ^ this creates a httpd server that listens on port 443.
>>
>> ServerAdmin uaf-libra...@alaska.edu 
>> DocumentRoot /dspace/webapps/xmlui
>> ServerName scholarworks.alaska.edu
>> ServerAlias scholarworks
>>
>> ErrorLog logs/scholarworks.alaska.edu-https-error_log
>> CustomLog logs/scholarworks.alaska.edu-https-access_log 
>> combinedssl
>>
>> Include conf.d/ssl.include
>> Include conf.d/ssl.include.star
>> ### ^ these point to a file which specifies where the ssl 
>> certificates
>> ###   live on the host.
>>
>> ProxyPass / http://127.0.0.1:8080/  retry=10 
>> connectiontimeout=5
>> ### ^ this tells httpd to redirect it's / to localhost port 8080
>>
>>
>
> Again, not a redirect; this causes HTTPD to send a request to whatever is 
> on port 8080 (Tomcat, I presume) 

Re: [dspace-tech] Tomcat Cert Process for Dspace

[wlrutherford]

Hello Tony. I work with Dayne and we are successfully using a localized 
version
of your example at our site - Thank you. But there's one problem.

We are using port 443 for secure connections and securing the site by 
rewriting
port 80 to also go to port 443. The handle server uses ports 2641 and 8000 
and
needs them to to be non-SSL. But something is converting our 
http://hdl.handle.net/
URI links to https. So, rather than linking to the document page, we get 
something
like this:



400 Bad Request

Bad Request
Your browser sent a request that this server could not understand.
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
Hint: https://scholarworks.alaska.edu/;>https://scholarworks.alaska.edu/

Apache/2.2.15 (Red Hat) Server at scholarworks.alaska.edu Port 
443



My best guess at the moment is that the virtualhost configurations are
also somehow catching other ports and rewriting them in the same way.


#
### Redirect insecure ports to secure port 443
ServerAdmin uaf-library-it-d...@alaska.edu
RewriteEngine On
RewriteCond %{HTTPS} off
#   RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]
RewriteRule (.*) https://%{HTTP_HOST}:443%{REQUEST_URI}
DocumentRoot /dspace/webapps/xmlui
ServerName scholarworks.alaska.edu
ServerAlias scholarworks

ErrorLog logs/scholarworks.alaska.edu-error_log
CustomLog logs/scholarworks.alaska.edu-access_log common

ProxyPass / http://127.0.0.1:443/  retry=10 connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 443

#timeout=300
ProxyPassReverse / http://127.0.0.1:443/  retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.



#


### ^ this creates a httpd server that listens on port 443.

ServerAdmin uaf-library-it-d...@alaska.edu
DocumentRoot /dspace/webapps/xmlui
ServerName scholarworks.alaska.edu
ServerAlias scholarworks

ErrorLog logs/scholarworks.alaska.edu-https-error_log
CustomLog logs/scholarworks.alaska.edu-https-access_log combinedssl

Include conf.d/ssl.include
Include conf.d/ssl.include.star
### ^ these point to a file which specifies where the ssl 
certificates
###   live on the host.

ProxyPass / http://127.0.0.1:8080/  retry=10 connectiontimeout=5
### ^ this tells httpd to redirect it's / to localhost port 8080

#timeout=300

ProxyPassReverse /  http://127.0.0.1:8080/  retry=10
### ^ this tells httpd that tomcat's url's should be rewritten to 
look
###   like they're coming from httpd.

ProxyPreserveHost On
### ^ this tells httpd to keep the Host: information from the 
client and
### pass it on to tomcat.

#SSLUseStapling on
### Added due to error when starting apache.  Dayne Ellanna



We once were trying to do this via Tomcat rewriting 8080 to 443. But when
we gave up on Tomcat I disabled the unneeded ports 8080 and 8443. I also
changed the proxy references to 8080 in  to 443; that 
won't
work in the  section because it would reroute to itself.

So why do I need the 443 proxy directives at all? I tried commenting out the
entire line but that just breaks the whole site. The only place ports 2641 
and
8000 are ever mentioned is in the handler-server/config.dct file.

{
"hdl_http_config" = {
"bind_address" = "137.229.115.121"
"num_threads" = "15"
"bind_port" = "8000"
"backlog" = "5"
"log_accesses" = "yes"
}

"server_type" = "server"
"hdl_udp_config" = {
"bind_address" = "137.229.115.121"
"num_threads" = "15"
"bind_port" = "2641"
"log_accesses" = "yes"
}

"hdl_tcp_config" = {
"bind_address" = "137.229.115.121"
"num_threads" = "15"
"bind_port" = "2641"
"backlog" = "5"
"log_accesses" = "yes"
}

"no_udp_resolution" = "n"
"interfaces" = (
"hdl_udp"
"hdl_tcp"
"hdl_http"
)

"server_config" = {
"server_admins" = (
"300:0.NA/11122"
)

"replication_admins" = (
"300:0.NA/11122"
)

"storage_type" = "CUSTOM"
"storage_class" = "org.dspace.handle.HandlePlugin"
"max_session_time" = "8640"
"this_server_id" = "1"
"max_auth_time" = "6"
"backup_admins" = (
"300:0.NA/11122"
)

"case_sensitive" = "no"
}

}


Does anyone see anything glaringly wrong with this setup?
I believe the local handle-server process passes the request
to handle.net just fine. But somewhere during the return trip
we're redirected to port 443 and things to south.

Thanks for any tips, insights, or corrections you can provide,

  Walter R.



On Friday, October 6, 2017 at 1:28:00 AM UTC-8, Tony Brian Albers wrote:
>
> On 10/06/2017 08:39 AM, Dayne Ellanna wrote: 
> > I am having the