OK so I just spent an age trying to push my project to a GitHub repositry but
I think i must just be too tired to use my brain correctly.
Eventally gave up and just pasted my servlet into a file there.
https://github.com/Leighbee13/RUNTHIS/tree/master
Let me know if this works!
>________________________________________
>From: Christopher Schultz <ch...@christopherschultz.net>
>Sent: 30 April 2014 20:13
>To: Tomcat Users List
>Subject: Re: Tomcat 8.0.3.0 getting never before seen by google Illegal State
>Exception. Sevlets outputting the >audio output from the previous runs of the
>program instead of the current run.
>-----BEGIN PGP SIGNED MESSAGE-----
>>Hash: SHA256
>Leigh,
>On 4/30/14, 1:55 PM, Hayward, Leigh wrote:
>> Hi Chris, Thank you so much for the speedy response . I have no
>> idea what i'm doing formatting wise. Pretty much just wrapping and
>> indenting all by hand so forgive me if it's a mess!
>>
>>> ________________________________________ From: Christopher
>>> Schultz <ch...@christopherschultz.net> Sent: 30 April 2014 17:58
>>> To: Tomcat Users List Subject: Re: Tomcat 8.0.3.0 getting never
>>> before seen by google Illegal State Exception. Sevlets outputting
>>> the >audio output from the previous runs of the program instead
>>> of the current run.
>>
>> Leigh,
>>
>> On 4/30/14, 10:31 AM, Hayward, Leigh wrote:
>>>> My Java EE web application takes in multiple audio inputs
>>>> and outputs them as a single wav file via an
>>>> application/octet stream.
>
> Like a mixer?
>
>> Like a concatenator, adds songs one after the other into one long
>> song. It then overlays (mixes) another file into it containing
>> audio cues on top of the concatenated music. But the issue is
>> definitely arising at the concatenation phase.
>Okay. Are you saying that you don't get the output file you are
>expecting given a set of input files, even when under test conditions
>(a single request)?
>Do you accept all files at once for upload and return a single
>concatenated file, or do you upload the files one at a time and then
>request the completed file at the end of the workflow?
Yes i'm not getting the output I expect given the inputs, i'm getting
the output that would have happened with a previous input.
Yes the form has an unknown number of file inputs and they are
all submitted at once.
>> I have the audio writing to a file after each song is added in
>> the concatenation phase so that i can audibly debug and the
>> concatenator is definitely struggling to find the uploaded
>> files.
>What do you mean "struggling to find the uploaded files"? Your file
>upload library should be readily giving the files to you.
By struggling I just mean that the concatenate method is where the
incorrect audio output begins being produced instead of the correct
one. At least as far as i can tell from my
>>>> It seemingly randomly works correctly (i.e. outputting the
>>>> correctly manipulated audio file) but sometimes, the file
>>>> from a previous run of the program is output and I get one of
>>>> these errors:
>>>>
>>>> SEVERE [http-nio-8084-exec-30]
>>>> org.apache.catalina.loader.WebappClassLoader.clearReferencesThreads
>>>>
>>>>
The web application [/MyApp] is still processing a request that has
>>>> yet to finish. This is very likely to create a memory leak.
>>>> You can control the time allowed for requests to finish by
>>>> using the unloadDelay attribute of the standard Context
>>>> implementation.
>>>>
>>>> and to me seemingly random numbers of these errors:
>>>>
>>>> "SEVERE [http-nio-8084-exec-87]
>>>> org.apache.coyote.http11.AbstractHttp11Processor.process
>>>> Error processing request java.lang.IllegalStateException: The
>>>> resources may not be accessed if they are not currently
>>>> started?"
>
>
> These kinds of things are almost always due to storing of a request
> or response option in some kind of structure that survives past the
> end of a particular request.
>
> Can you explain how you build the response -- specifically
> involving any non-standard threading you may do?
>
>> The response is built in a method called finish which is called
>> in doPost like this :
>
>> finish(req, res); In finish I make a printwriter I set response
>> content type to "application/octet-stream", set the header
>> "Content-Disposition" to "attachment; filename=finished.wav" I
>> make a FileInputStream of the finished filepath Then I set the
>> header content length to the length of the finished file. then I
>> while loop the fileInputStream into the printwriter Then I close
>> the FIS.
>That seems fairly straightforward. Why are you bothering with a
>PrintWriter? The existing response OutputStream ought to be good
>enough to pump bytes through. How are you pumping the bytes? Just a
>simple, locally-declared byte array?
The online examples use printwriters so I used a printwriter.
I've now changed it to the res's own output stream but it made no
difference to the output.
I'm just using:
while ((i = fileInputStream.read()) != -1) {
out.write(i);
}
fileInputStream.close();
out.close();
>You should remember to flush and/or close the output stream as well. I
>think Tomcat will flush and close this for you, but you might want to
>check to see if it changes anything
.
Sorry i did close the printwriter as well just forgot to mention it!
I'm now flushing it too. but it makes no difference.
>> I have not touched threading since it looks terrifying. I did
>> experiment with making the servlet singlethreaded since I thought
>> it could have something to do with threads (bad practice I know)
>> but that did nothing to help.
>Are you using any variables that have been declared at the class
>level? Everything you do with a request should be declared inside of
>the service method (or doPost, etc.) or a parameter passed-into a
>utility method. Don't store references to request, response, etc.
>outside of the thread of execution that is handling that request.
>If single-threading fixes your problem (sounds like it doesn't), then
>you have broken the above rule.
Yeah I read this article
http://www.javaworld.com/article/2072798/java-web-development/write-thread-safe-servlets.html
And removed all of my variable declared outside their respective
classes and it didn't help, then I single-threaded it and that didn't
work either. Does that rule out threads as being the potential
problem?
>>>> The files always upload correctly to my filesystem, but
>>>> something is going wrong when I try to access them in order
>>>> to process them.
>
> What mechanism do you use for upload?
>
>> My servlet has a location set in the @multipartconfig tag that is
>> where all files are saved to and retrieved from. I am getting the
>> Parts from the request then iterating through them, for each part
>> that is a file upload I use part.write("uploadedfile" + count +
>> ".wav");
>In that case, the files should all have arrived on the disk before
>your code gets a chance to read them. Remember that they aren't
>required to be on the disk: you should be reading them from the
>InputStream(s) that the container (Tomcat) gives you. Also remember
>that the container (Tomcat) will probably delete those files after the
>request has completed, so you can't rely on them being in any
>particular place for any amount of time: if you want a copy of a file,
>you'll have to make it yourself.
What do you mean by not required to be on the disk? As in I should be
copying files into that file again? Nothing appears to be being deleted
after I run it. When I go to the sound file, all the files are still there
after the run is over.
>>>> Also when it is downloading the file it appears to the user
>>>> to be several MB long despite the file that is output being
>>>> only a few thousand KB.
>
> NB a few thousand KiB is the definition of several MiB.
>
>> I am possibly just an idiot on that one. I think the deadline
>> sleep deprivation is getting to me
>It happens to all of us.
Glad to hear it!
> Is the response built before you stream any of it to the client?
> Are you setting a Content-Length before you send any data? Are you
> using chunked responses?
>
>> I call a finish(req, res) method in my doPost which is where all
>> my response building is done. I am setting a content length
>> before I send the data because I wanted to be able to see how
>> long it was going to take to download since the program is fairly
>> slow.
>It doesn't seem like concatenating files would take that much time,
>besides the upload time. I don't know much about WAV files, though...
>perhaps you have to do the math to connect them together properly. If
>you have to re-encode entire files, though, you are probably doing it
>wrong.
Oh I am most certainly doing something wrong! I'm not too concerned
with speed right now i just want it to output the right thing.
To connect them I'm just using a SequenceInputStream.
See the GitHub.
>>>> Sometimes when it doesn't work it does not produce these
>>>> errors, but it never produces these errors when it works
>>>> correctly.
>
> Does (increased) load seem to make the situation worse?
>
>> Definitely. Always happens with long audio files and only
>> sometimes with shorter test ones.
>So long files end up breaking things more than short files? When I
>said "load", I really meant "numbers of simultaneous requests"
>(increasing).
There will never be more than one user as it will never really go
online.
>If you can't even handle one isolated request with long files, you
>have a serious problem.
Tell me about it. Honestly as long as I eventually get the output
I'm looking for I don't mind.
>>>> I have googled it but there's no reference to the second kind
>>>> of error anywhere on the web aside from svn commits by
>>>> tomcat developers, so while I am a total newbie to mailing
>>>> lists, after exhausting stackoverflow this seemed like the
>>>> only place to turn to. I'm developing a java EE web
>>>> application in Netbeans using Tomcat 8.0.3.0 on a windows 7
>>>> operating system.
>
> (You should upgrade to a later Tomcat 8 beta. Nothing should
> affect what you are seeing here, but it's not a bad idea to get the
> latest.)
>
>> I'm just a little nervous about moving or touching anything since
>> i'm not 100% confident it wont just break everything. Still new
>> at this. I wanted the most stable version, to balance out my
>> intellectual instability.
>The most stable version of 7.0.53. The 8.0.x line hasn't yet been
>voted stable, though it pretty much is. Latest 8.0.x is 8.0.5 and will
>soon be 8.0.6, but it has only been voted beta at this point.
Do I have to? I know you're the expert and I trust that what you're
saying is completely correct and all I just really don't want to mess
with things too much this late in the game.
>>>> The web application is very basic and allows users to upload
>>>> files via a multipart html form. This then posts to a servlet
>>>> which first uploads these files to the programs file system
>>>> and then accesses them, concatenates them together and saves
>>>> them back on the file system. It's better explained by this
>>>> diagram http://imgur.com/Oacd4gq
>
> The real question is whether you are using non-request-processor
> threads to do any of this work. Also, where are the various data
> stored while you are working on them? Not the filesystem -- that
> seems self-evident -- but where in the program are the String[]
> objects that represent all the songs that have been uploaded?
>
>> C:\Users\Leigh\Documents\NetBeansProjects\MyApp\src\java\sound
>
>> All audio files get stored in and are accessed from there.
>No, somewhere in your program you have a String[] declared where you
>have a reference to those filenames. You shouldn't actually have
>references to path names at all, because the container can buffer the
>files in memory instead of ever writing them to the disk.
Ah sorry, the string [] songs gets created in doPost by having a counter
in my parts iterator that counts how many audio files there are and
then a string [] is created of the song file locations them using a for
loop. It is then put into the overlay method as a variable.
>Anyhow, I was wondering if you were using a String[] declared at the
>servlet level to track this stuff. That always results in disaster,
>because multiple requests will fight over the same variable and step
>on each other's toes.
>Any chance you could post your code somewhere for inspection? Dropping
>it right into a post to the list is just fine.
Done :)
>> This is purely a dissertation project so I have no intention of
>> making it work for multiple users or in fact be functional
>> anywhere outside of my laptop. All files are being uploaded to
>> the file system of the project itself since that was the only way
>> I knew how to make it work.
>Why bother with a web application interface, then?
Because this would, in the world that is the context of my
dissertation, be a web application. I don't need it to be
one in the real world. I just need to display that can use
technologies like CSS, HTML and Servlets in order to get
a good grade.
> When you process a "transaction" (upload -> merge -> respond), are
> you taking care to ensure that the files are placed into different
> paths for each request? How are you choosing the filename of the
> merged audio clip?
>
>> all uploads are called uploadedfile1, uploadedfile2,
>> uploadedfile3 etc depending on how many files are uploaded.
> If these files all go to the same place, then multiple requests will
> overwrite each other. That's going to be a problem, except you said
> you don't care. Honestly, you should either care about this and make
> it work properly, or just drop the "web" requirement where it's okay
> to be sloppy.
It's not that I don't care and i can't drop the web requirement as it's a
university project. Again, there ill never be multiple requests it's a
purely theoretical project. All I want is for it to output the files that i
am inputting concatenated and overlaid.
>> When these files don't exist already on my file system I
>> sometimes get null pointer errors and an empty output.
>> Overwriting works better than creating new.
>
>> The filename of the finished file is just called finished.wav.
>> Always. It just gets overwritten each time I run it. Or not as it
>> seems.
I would choose a different filename every time. Especially if it looks
like the old copy of the file hasn't been closed entirely, yet.
>>>> Could it be that the files are not being uploaded fully
>>>> before they are being accessed?
>
> I don't believe this is possible when using Tomcat's file-upload
> implementation. It is certainly possible to do if you are
> performing the multipart handling yourself. What tool are you using
> to fetch the multipart data from the request?
>
>> As I said above I just get the parts from the request, for loop
>> through them and write any audio files to the locations specified
>> in my MultipartConfig.
Looking at the code would be very helpful. Feel free to leave-out any
cutting-edge research-y stuff that you don't want to divulge ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJTYUtlAAoJEBzwKT+lPKRYrtsQAKaQHBjvC8i4UJngL3mDtGMD
t3JxMET3Nb/AGAkeBFn2yJR+PTR4Vuj1gbEKrHN+SE3l1VlbVmjxfOQlPDAm3SPF
SE50zTnmYKKdcOMHpIuUeiMgJh/uKCym+bp3PNcuvORZq/FVJdJ3b8me0MQUO3Vh
juCqj/h/UV+eKZaFmLA+dcJAQoxmHgVoCidnwRN255LUve8/o/JQXthTjmCGuf4Z
rQ7ZarxVr8wbXZJ5wQZ/O1Z6K7IIZ07hNbDHCtj+3ZEeEcSzn5IEOMpskhSu48bC
5LTqn5gXgE7m+uFszjjpshRhoO2PPRf2sb39IYVkkExqVb5EOkdNpVVckRaBd371
H14S/CoWQcCvfq+QORl/i95vrMf+P7mMqD+wS3Bagyw3zD0spTu48oLcyG2Crmwd
zCFyezw3SIJdis8yQBsl7uGNMELF5gJ6n0oPvcUwjiY7tHbZ6f6eF5aEPAz/r1eS
uf3OHeqJswRW9qBUGPxEtvNOajs4q0o6BacfQpiQlHeLNzh6OzGsLmzUZJGoWiPV
zJR+52Z+5RFXDI38E+XSX/T9UFF7M1WjH0mfjpYz6TjIDZl7H+FH7Od5+tQ02M4C
tlr66gq2HqpQA1SlhnRpIpSsyF61PIJIteX1rcRW6qik6qizn4XJQ8GoweCwV8Jn
F2ASYC6UsDVoHpHtJfrg
=uGx4
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
