4-24MB XML? Ah well... On the IDE side: if you open a file as textfile (instead of marked up whatever, XML in your case) things 'usually' work. Editor might still be too slow, but it's something to try. Either convince the IDE somehow (open as..), otherwise either create a symbolic link or rename temporarily (if on Windows and no funky tools installed to manage links).
On the code side: I don't know if you have any problem with serving/ processing the XML file, but if you do: in general the problem are the DOM trees/parsers, because you might end up with reading the whole thing into memory before starting to process. If you use tools that are SAX based or support pull parsing, you can stream the file, or just load the parts that are interesting respectively (or if it's even heavier: look at Xindice, an XML DB written in Java). Sidenote on the filesize: I don't know about you, but I usually end up only with files this big, if a file was the result of a computation or transformation. Usually such files are highly redundant, and/or act like a database. So you might want to muck around a bit with your data representation to have it come down in size, or split it to more files (xml include) and end up being more manageable for you and for the applications involved. YMMV. If you *really* need a file that size, and need it transmitted to the user, be sure to compress it only once with gz and then serve that directly. Nginx has options that can help you there, since it was written to serve massive data streams (assuming that you want to use Nginx in front of your container of choice, same can be achieved in your container directly with a bit of effort). And if you don't need to transmit it to users, but cannot reduce it otherwise, you could actually store it in an XML binary format (like Fast Infoset (FI) or Efficient XML Interchange (EXI)). That would at least allow for a read speed improvement (but you'll not be able to edit by hand anymore). Hope some of this helps, Daniel Some pointers for the interested: XML Pull Parsing: http://www.extreme.indiana.edu/xgws/xsoap/xpp/ Comparison of DOM libs: http://www.dom4j.org/dom4j-1.6.1/compare.html (links at the end of the page are quite old but interesting in terms of performance) XML DB: http://xml.apache.org/xindice/ Fast Infoset (ITU-T standard): http://en.wikipedia.org/wiki/Fast_Infoset (http://java.sun.com/developer/technicalArticles/xml/fastinfoset/, http://fi.dev.java.net/) Efficient XML Interchange (currently W3C draft): http://en.wikipedia.org/wiki/EXI (http://exificient.sourceforge.net/) FI (and EXI soon) is used in Metro (Glassfish's WS stack): https://metro.dev.java.net/ (http://wiki.apache.org/ws/ StackComparison) On Apr 23, 12:12 am, jsm2prof <jsm2p...@googlemail.com> wrote: > Dave, > Thanks for your suggestions. Maybe when you understand what I am > trying to do you will have a different suggestion. > > On Apr 22, 10:45 am, David Pollak <feeder.of.the.be...@gmail.com> > wrote: > > > On Tue, Apr 21, 2009 at 10:15 PM, jsm2prof <jsm2p...@googlemail.com> wrote: > > > > Netbeans 6.5 gave me an Out of Memory Error when I loaded a 4MB > > > file. > > > If you've got a 4MB scala source code file, I'd suggest refactoring a little > > bit. > > 26 years ago when I was as young and smart as you are now, I > refactored a program that only ran on a multi-million dollar computer > so that it ran on a home assembled floppy based desktop. But, I do > not think refactoring would be the best solution now. > > > > > If it's a 4MB text file, you'd likely be better served with vi or Emacs > > anyway... they're much better at dealing with large files (especially vi). > > I use these a little but it is XML not really text and I do not want > to encourage users to use these tools in order to find information on > the web. > > > > Actually, I also have a 24 MB XML data file that I would like to > include in a Scala program. Once this works, then I would like to > publish it on a website as FLOSS so that anyone can see quick and easy > answers to any arbitrary thing they might want to know about this > data. After that is done then I want to do it again several times > with the same data in different languages. > > I believe I can encourage Scala/Lift usage by doing it this way and > eventually giving instructions so that beginners learn enough Scala/ > Lift to do their searches easier than they could have done the search > with store bought software. > > > > I get much better help from Eclipse. Scala is tops. Lift > > > seems to be very good. I only want to learn the things that are > > > best. Would someone please kick Lift up to the status of playing > > > with the heavy weight? > > > Lift is a web framework. While we give input and feedback to the IDE > > maintainers (and they give us great support), the use of Lift is independent > > of any IDE. Perhaps you'd be best served by seeing what groups support > > NetBeans for non-Scala-related file types (once again, if you've got a 4MB > > Scala file, there's some work that you need to do in terms of refactoring > > before asking for support). > > Oracle just bought SUN. I remember seeing Oracle in times past > pushing Eclipse. I do not remember seeing Oracle in times past > pushing Netbeans to the same extent. Eclipse does Python and many > other things that people used before we ever even heard of Scala/ > Lift. There are a some of us who learned some of Netbeans just in > order to do Scala/Lift. There are many more who just would not learn > another IDE just for Scala/Lift. > > I did not care for all of the extras necessary to use Java. But when > Scala/Lift became available, I learned Maven because that was the only > way I could get a tiny web page to work with Lift. I really had high > hopes that I could just use just one IDE, one programming language > (Scala), and one web framework (Lift). I have learned hundreds of > different languages, compilers, editors, environments, and extras. I > am sick and tired of it. > > Please do whatever you can to move the largest Lift projects away from > multiple extras. > > Thanks, > A lower level teacher > > > > > Thanks, > > > David --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~----------~----~----~----~------~----~------~--~---
