[android-beginners] Re: Poor performance on my file loader (code optimization help!)
On Sep 20, 11:28 am, Glen Kimsey wrote: > The loading process on a sample (1000+ line) file is abysmally slow > (more than 4 minutes). [...] > > Some possible causes for the slowness: > - Extensive use of parseFloat() > - Creating new StringTokenizer for every line > - Inefficiencies in for() loops (not caching as necessary?) > - Repeated use of the acos() and sqrt() functions Re-use objects when possible to avoid allocation and construction overhead. For a parser using regular expressions, you avoid the cost of recompiling the RE. acos() and sqrt() are not especially fast; sqrt () in particular, since the G1 lacks floating-point hardware. You can get some sense for where the time is going with the method tracing facility: http://developer.android.com/guide/developing/tools/traceview.html > [...] I could even do something as drastic as > performing this load in a separate app (before distribution) and > somehow exporting the resulting object (KdfAnimation) that could more > easily be picked back up (this would work in C, but I'm doubtful Java > would allow such low-level memory copying). Java does provide a serialization mechanism, but it's not widely loved. :-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Android Beginners" group. To post to this group, send email to android-beginners@googlegroups.com To unsubscribe from this group, send email to android-beginners-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-beginners?hl=en -~--~~~~--~~--~--~---
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
On 20 Wrz, 20:28, Glen Kimsey wrote:
> Hello!
> I'm fairly new to both Android and Java (pretty experienced in C/C++),
> and (stupidly, perhaps) took on a big project. In it, I need to load
> information about 3D models and their respective animations from a
> text file (proprietary file format based on PSK/PSA).
>
> private static float[] parseRotLine(String line) {
> float[] retFloat = new float[4];
> StringTokenizer tok = new StringTokenizer(line);
>
> // Ignore "rot"
> tok.nextToken();
> // theta
> tok.nextToken();
> retFloat[0] = wToTheta(Float.parseFloat(tok.nextToken()));
> // x
> tok.nextToken();
> retFloat[1] = Float.parseFloat(tok.nextToken());
> // y
> tok.nextToken();
> retFloat[2] = Float.parseFloat(tok.nextToken());
> // z
> tok.nextToken();
> retFloat[3] = Float.parseFloat(tok.nextToken());
>
> // Need to scale this vector to make glRotate happy
> retFloat = scaleAxisAngleVector(retFloat);
>
> return retFloat;
> }
>
> private static float wToTheta(float w) {
> // w = cos(theta/2)
> float theta = (float)Math.acos((double)w) * 2;
> // convert theta to degrees
> theta = 180 * theta / (float)Math.PI;
> // The above calculation only gave us a value from 0 to 180
> because
> of arccosine
> // TODO: Determine if this is a problem. Note that w can be
> negative, and this implies 180 degrees
> if (w < 0) {
> theta += 180;
> }
> return theta;
> }
>
> private static float[] scaleAxisAngleVector(float[] unscaled) {
> float[] scaled = new float[4];
> // scale = sqrt(x^2 + y^2 + z^2);
> float scale = (float)Math.sqrt(unscaled[1] * unscaled[1] +
> unscaled
> [2] * unscaled[2] + unscaled[3] * unscaled[3]);
>
> if (scale != 0) {
> scaled[0] = unscaled[0];
> scaled[1] = unscaled[1] / scale;
> scaled[2] = unscaled[2] / scale;
> scaled[3] = unscaled[3] / scale;
> }
> else
> {
> // Scale can be 0, which means we're not rotating
> // but dividing by zero is bad, and glRotate still
> // needs a normalized vector.
> scaled[0] = 0;
> scaled[1] = 1;
> scaled[2] = 0;
> scaled[3] = 0;
> }
> return scaled;
> }}
>
> End of Loader Code
Ugh, why oh why are you transforming quaterninions to axis/angle
representation? Quaternion interpolation is easier, and gives better
result (you will also save a lot of time on calculations).
About performance, if it's possible (eg. you are not downloading the
scene files from network or smth), write a small utility that will
convert text file to binary representation offline, and then read the
binary files on android. Also try to minimize number of allocation
during file loading.
--
Bart 'arcone1' janusz
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Android Beginners" group.
To post to this group, send email to android-beginners@googlegroups.com
To unsubscribe from this group, send email to
android-beginners-unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/android-beginners?hl=en
-~--~~~~--~~--~--~---
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
Thanks for the reply. I think your suggestion of going binary is one
of the better ones I had considered. Especially because I can avoid
the parseFloat() calls altogether using this method.
As for the quaternion vs axis/angle, I had thought I HAD to translate
them originally (I found out later with more research that I just had
to use the more complicated matrix math rather than simple glrotate
and gltranslate calls if I didn't). I don't suppose you could direct
me to a tutorial or good description that will help a 3D rendering
newbie like me use the quaternions? If it really is better/faster, I
want to go that route rather than finding out later I should have.
I'm still looking for any additional suggestions on what particular
parts of the code might be killing my load times, although I'm sure
switching to binary and not using acos and sqrt will help quite a bit.
Thanks,
Glen
On Sep 22, 5:03 am, a1 wrote:
> On 20 Wrz, 20:28, Glen Kimsey wrote:
>
> > Hello!
> > I'm fairly new to both Android and Java (pretty experienced in C/C++),
> > and (stupidly, perhaps) took on a big project. In it, I need to load
> > information about 3D models and their respective animations from a
> > text file (proprietary file format based on PSK/PSA).
>
>
>
>
>
>
>
> > private static float[] parseRotLine(String line) {
> > float[] retFloat = new float[4];
> > StringTokenizer tok = new StringTokenizer(line);
>
> > // Ignore "rot"
> > tok.nextToken();
> > // theta
> > tok.nextToken();
> > retFloat[0] = wToTheta(Float.parseFloat(tok.nextToken()));
> > // x
> > tok.nextToken();
> > retFloat[1] = Float.parseFloat(tok.nextToken());
> > // y
> > tok.nextToken();
> > retFloat[2] = Float.parseFloat(tok.nextToken());
> > // z
> > tok.nextToken();
> > retFloat[3] = Float.parseFloat(tok.nextToken());
>
> > // Need to scale this vector to make glRotate happy
> > retFloat = scaleAxisAngleVector(retFloat);
>
> > return retFloat;
> > }
>
> > private static float wToTheta(float w) {
> > // w = cos(theta/2)
> > float theta = (float)Math.acos((double)w) * 2;
> > // convert theta to degrees
> > theta = 180 * theta / (float)Math.PI;
> > // The above calculation only gave us a value from 0 to 180
> > because
> > of arccosine
> > // TODO: Determine if this is a problem. Note that w can be
> > negative, and this implies 180 degrees
> > if (w < 0) {
> > theta += 180;
> > }
> > return theta;
> > }
>
> > private static float[] scaleAxisAngleVector(float[] unscaled) {
> > float[] scaled = new float[4];
> > // scale = sqrt(x^2 + y^2 + z^2);
> > float scale = (float)Math.sqrt(unscaled[1] * unscaled[1] +
> > unscaled
> > [2] * unscaled[2] + unscaled[3] * unscaled[3]);
>
> > if (scale != 0) {
> > scaled[0] = unscaled[0];
> > scaled[1] = unscaled[1] / scale;
> > scaled[2] = unscaled[2] / scale;
> > scaled[3] = unscaled[3] / scale;
> > }
> > else
> > {
> > // Scale can be 0, which means we're not rotating
> > // but dividing by zero is bad, and glRotate still
> > // needs a normalized vector.
> > scaled[0] = 0;
> > scaled[1] = 1;
> > scaled[2] = 0;
> > scaled[3] = 0;
> > }
> > return scaled;
> > }}
>
> > End of Loader Code
>
> Ugh, why oh why are you transforming quaterninions to axis/angle
> representation? Quaternion interpolation is easier, and gives better
> result (you will also save a lot of time on calculations).
>
> About performance, if it's possible (eg. you are not downloading the
> scene files from network or smth), write a small utility that will
> convert text file to binary representation offline, and then read the
> binary files on android. Also try to minimize number of allocation
> during file loading.
>
> --
> Bart 'arcone1' janusz
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google
Groups "Android Beginners" group.
To post to this group, send email to android-beginners@googlegroups.com
To unsubscribe from this group, send email to
android-beginners-unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/gr
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
Fadden, Sorry, I missed your reply when I responded to Bart! I tried to find significant ways that I could avoid allocation, but the StringTokenizer is the main thing that I'm calling 'new' on and I didn't see any way to avoid doing that for every line. Your suggestion of using regexs for a parser kind of threw me off, because I didn't even see that as a possibility. I'll have to look into that and see what you mean. I also didn't know the G1 didn't have floating- point hardware. Will this likely be the case for other android phones in the near future? This information combined with Bart's suggestion to stick with quaternions means I'll very likely be removing those calls to acos and sqrt entirely. Also, thanks for the links. Your reply was extremely helpful. Thanks! Glen On Sep 21, 7:22 pm, fadden wrote: > On Sep 20, 11:28 am, Glen Kimsey wrote: > > > The loading process on a sample (1000+ line) file is abysmally slow > > (more than 4 minutes). [...] > > > Some possible causes for the slowness: > > - Extensive use of parseFloat() > > - Creating new StringTokenizer for every line > > - Inefficiencies in for() loops (not caching as necessary?) > > - Repeated use of the acos() and sqrt() functions > > Re-use objects when possible to avoid allocation and construction > overhead. For a parser using regular expressions, you avoid the cost > of recompiling the RE. acos() and sqrt() are not especially fast; sqrt > () in particular, since the G1 lacks floating-point hardware. > > You can get some sense for where the time is going with the method > tracing facility: > > http://developer.android.com/guide/developing/tools/traceview.html > > > [...] I could even do something as drastic as > > performing this load in a separate app (before distribution) and > > somehow exporting the resulting object (KdfAnimation) that could more > > easily be picked back up (this would work in C, but I'm doubtful Java > > would allow such low-level memory copying). > > Java does provide a serialization mechanism, but it's not widely > loved. :-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Android Beginners" group. To post to this group, send email to android-beginners@googlegroups.com To unsubscribe from this group, send email to android-beginners-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-beginners?hl=en -~--~~~~--~~--~--~---
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
On Sep 23, 8:49 am, Glen Kimsey wrote: > I tried to find significant ways that I could avoid allocation, but > the StringTokenizer is the main thing that I'm calling 'new' on and I > didn't see any way to avoid doing that for every line. Your > suggestion of using regexs for a parser kind of threw me off, because > I didn't even see that as a possibility. I'll have to look into that > and see what you mean. >From the JavaDoc for StringTokenizer: "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead." > I also didn't know the G1 didn't have floating- > point hardware. Will this likely be the case for other android phones > in the near future? This information combined with Bart's suggestion > to stick with quaternions means I'll very likely be removing those > calls to acos and sqrt entirely. It also lacks an integer divide instruction, so go easy on divisions and modulus. :-) If you do a google search for "ODROID" you can see an example of a device with hardware FP. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Android Beginners" group. To post to this group, send email to android-beginners@googlegroups.com To unsubscribe from this group, send email to android-beginners-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-beginners?hl=en -~--~~~~--~~--~--~---
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
Hi, > As for the quaternion vs axis/angle, I had thought I HAD to translate > them originally (I found out later with more research that I just had > to use the more complicated matrix math rather than simple glrotate > and gltranslate calls if I didn't). I don't suppose you could direct > me to a tutorial or good description that will help a 3D rendering > newbie like me use the quaternions? If it really is better/faster, I > want to go that route rather than finding out later I should have. Quaternions aren't faster (well ok nlerp can be) but they are better at least for interpolation, sice you are clearly reading keyframing data, thus I assumed that you will interpolate it sooner or later therefore transforming them to axis/angle looks a bit redundant (since usually rotation is interpolated with cubic quaternion interpolation aka. squad). If yor some basic info about quaternions you can check this old (very) flipcode FAQ: http://www.flipcode.com/documents/matrfaq.html (this is really basic so may be a bit too basic :)) After bit searching I also found this: www.itu.dk/people/erikdam/DOWNLOAD/98-5.pdf looks ok to me And of course if you have Foley, van Dam "CG: Principles And Practice" book. (It may be a bit outdated nowadays but still it's a CG bible) -- Bart 'arcone1' Janusz --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Android Beginners" group. To post to this group, send email to android-beginners@googlegroups.com To unsubscribe from this group, send email to android-beginners-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-beginners?hl=en -~--~~~~--~~--~--~---
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
Is there a reason to why you use a LineNumberReader? I would instead
use a BufferedReader, that will probably improve your performance.
I don't know anything about the dataformat in the file, but there
probably is a much faster way to parse it than using all those
StringTokenizers (or regexp even if that might be faster). I would
also try to parse the file without mark and reset.
On 20 Sep, 20:28, Glen Kimsey wrote:
> Hello!
> I'm fairly new to both Android and Java (pretty experienced in C/C++),
> and (stupidly, perhaps) took on a big project. In it, I need to load
> information about 3D models and their respective animations from a
> text file (proprietary file format based on PSK/PSA).
>
> The loading process on a sample (1000+ line) file is abysmally slow
> (more than 4 minutes). The good news is that there are lots and lots
> of ways I can improve it. What I'm hoping to find out here is if
> there's anything in *particular* in my code that is very inefficient
> that I can change. Don't worry about offending me, I know this is
> amateur code here.
>
> Some possible causes for the slowness:
> - Extensive use of parseFloat()
> - Creating new StringTokenizer for every line
> - Inefficiencies in for() loops (not caching as necessary?)
> - Repeated use of the acos() and sqrt() functions
>
> I'd like to find out from you gentlemen (ladies) what you think is the
> major thing eating up my time here. If there's some tool I can use to
> find out what lines of code are taking up the most time that'd be
> ideal. I'm open to any possible solutions. The file format can
> change (and probably will). I could even do something as drastic as
> performing this load in a separate app (before distribution) and
> somehow exporting the resulting object (KdfAnimation) that could more
> easily be picked back up (this would work in C, but I'm doubtful Java
> would allow such low-level memory copying).
>
> One final note is that the "action" loading section of this code
> occupies about 3/4 of the files being loaded (so this section may be
> more important to optimize).
>
> Loader Code
> public class KdfLoader {
>
> public static KdfAnimation load(String file) throws IOException {
> KdfAnimation anim = new KdfAnimation();
>
> FileInputStream fis = new FileInputStream(file);
> String line = "[start]";
> LineNumberReader lnr = new LineNumberReader(new
> InputStreamReader
> (fis));
>
> int boneNum = 0;
>
> try {
>
> for (line = lnr.readLine(); line != null; line =
> lnr.readLine()) {
> if (line.length() > 0) {
> if (line.startsWith("numbones")) {
> // Ignore it for now
> }
> else if (line.startsWith("bone")) {
> // Bone define
> StringTokenizer tok = new
> StringTokenizer(line);
> KdfBone bone = new KdfBone();
>
> // Toss out "bone"
> tok.nextToken();
>
> // Set name
> bone.boneName = tok.nextToken();
>
> if (boneNum == 0) {
> // Root bone. Ignore
> children and parent id in file
> bone.parentId = 0;
> bone.parent = null;
> // ignore "numchildren x"
> // ignore "parentid x"
> }
> else
> {
> // ignore "numchildren x"
> tok.nextToken();
> tok.nextToken();
> // ignore "parentid" text
> tok.nextToken();
> bone.parentId =
> Integer.parseInt(tok.nextToken());
> bone.parent =
> anim.allBones.get(bone.parentId);
> }
> // get loc and rot
> line = lnr.readLine();
> bone.loc = parseLocLine(line);
> line = lnr.readLine();
> bone.rot = parseRotLine(line);
>
> // Handle
[android-beginners] Re: Poor performance on my file loader (code optimization help!)
Thanks to everyone for the help on this. As I mentioned, I'm rather
new to Java, so things like StringTokenizer being legacy are news.
Believe it or not the file format uses actual frames instead of
keyframes (hence no timestamp or other indication of time or frames
between each being parsed out), but this is all still up in the air.
Obviously it's a very amateur way to do it right now. I'll probably
shift back to quaternions and worry about interpolating later (it
might be a application size vs. performance vs. memory usage thing in
the end. I'm not sure). Thanks for the links.
The LineNumberReader was actually hijacked from some open source code
intended for a very similar purpose. I didn't know what to use for
this scenario and so finding someone else using LNR meant I jumped on
it as the "ideal" solution. Safe to say I'll be trying R.E.s and/or
buffered reader soon enough. Are mark and reset really that taxing?
More so than basically buffering the several lines that might need to
be re-parsed?
Thanks again. Very helpful.
Glen
On Sep 24, 10:14 am, Kaj Bjurman wrote:
> Is there a reason to why you use a LineNumberReader? I would instead
> use a BufferedReader, that will probably improve your performance.
>
> I don't know anything about the dataformat in the file, but there
> probably is a much faster way to parse it than using all those
> StringTokenizers (or regexp even if that might be faster). I would
> also try to parse the file without mark and reset.
>
> On 20 Sep, 20:28, Glen Kimsey wrote:
>
> > Hello!
> > I'm fairly new to both Android and Java (pretty experienced in C/C++),
> > and (stupidly, perhaps) took on a big project. In it, I need to load
> > information about 3D models and their respective animations from a
> > text file (proprietary file format based on PSK/PSA).
>
> > The loading process on a sample (1000+ line) file is abysmally slow
> > (more than 4 minutes). The good news is that there are lots and lots
> > of ways I can improve it. What I'm hoping to find out here is if
> > there's anything in *particular* in my code that is very inefficient
> > that I can change. Don't worry about offending me, I know this is
> > amateur code here.
>
> > Some possible causes for the slowness:
> > - Extensive use of parseFloat()
> > - Creating new StringTokenizer for every line
> > - Inefficiencies in for() loops (not caching as necessary?)
> > - Repeated use of the acos() and sqrt() functions
>
> > I'd like to find out from you gentlemen (ladies) what you think is the
> > major thing eating up my time here. If there's some tool I can use to
> > find out what lines of code are taking up the most time that'd be
> > ideal. I'm open to any possible solutions. The file format can
> > change (and probably will). I could even do something as drastic as
> > performing this load in a separate app (before distribution) and
> > somehow exporting the resulting object (KdfAnimation) that could more
> > easily be picked back up (this would work in C, but I'm doubtful Java
> > would allow such low-level memory copying).
>
> > One final note is that the "action" loading section of this code
> > occupies about 3/4 of the files being loaded (so this section may be
> > more important to optimize).
>
> > Loader Code
> > public class KdfLoader {
>
> > public static KdfAnimation load(String file) throws IOException {
> > KdfAnimation anim = new KdfAnimation();
>
> > FileInputStream fis = new FileInputStream(file);
> > String line = "[start]";
> > LineNumberReader lnr = new LineNumberReader(new
> > InputStreamReader
> > (fis));
>
> > int boneNum = 0;
>
> > try {
>
> > for (line = lnr.readLine(); line != null; line =
> > lnr.readLine()) {
> > if (line.length() > 0) {
> > if (line.startsWith("numbones")) {
> > // Ignore it for now
> > }
> > else if (line.startsWith("bone")) {
> > // Bone define
> > StringTokenizer tok = new
> > StringTokenizer(line);
> > KdfBone bone = new KdfBone();
>
> > // Toss out "bone"
> > tok.nextToken();
>
> > // Set name
> > bone.boneName = tok.nextToken();
>
> > if (boneNum == 0) {
> > // Root bone. Ignore
> > children and parent id in file
> > bone.parentId = 0;
> > bone.parent = null;
> >
