Any reason why you use mawk instead of just regular awk? Impressive command :-).
+1 for opening JIRAs and fixing them for the next update. Sanjiva. On Sun, Jun 12, 2011 at 6:16 PM, Senaka Fernando <[email protected]> wrote: > > > On Sun, Jun 12, 2011 at 5:15 PM, Senaka Fernando <[email protected]> wrote: > >> >> >> On Sun, Jun 12, 2011 at 5:03 PM, Senaka Fernando <[email protected]> wrote: >> >>> FYI, run the following script inside $CARBON_HOME, "for f in `find -name >>> '*.jar' | xargs ls -l | mawk '{print $5" "$8}' | sort -gr | mawk '{print >>> $2}'`; do ls -lh $f; done;" >>> >>> The greatest contributor to the increase is size is Shindig with 16MB. If >>> you open up this bundle, you'll see that it embeds several jars that are >>> already available as dependencies. >>> >>> Ex:- wstx-1.0.7.jar, xercesImpl-2.9.1.jar, xml-apis-1.3.04.jar, >>> jdom-1.0.0.wso2v1.jar, httpclient-4.0.1.jar, icu4j-3.8.jar, jstl-1.2.jar >>> >>> This itself totals to around 7MB, which means that we could easily build >>> a 9MB shindig bundle. >>> >>> The second largest is >>> ./repository/components/plugins/axis2-1.6.1.wso2v1.jar. In here we embed >>> xmlbeans-2.3.0.jar as a jar, and also as a set of classes. This means that >>> we have two copies of xmlbeans-2.3.0.jar inside axis2-1.6.1.wso2v1.jar. >>> Also, xmlbeans-2.3.0.jar is found inside plugins >>> (./repository/components/plugins/xmlbeans-2.3.0.wso2v1.jar). Had we used >>> proper dependencies, the reduction in size of the axis2-1.6.1.wso2v1.jar >>> would be 5.2MB. >>> >>> 3rd Largest is Apache DS, and from the 7.2MB more than 1MB is coming from >>> commons-* bundles that have been embedded inside the >>> apacheds-1.5.7.wso2v1.jar. 4th in-line is apacheds-1.5.7.wso2v1.jar, >> >> >> Correction: 4th in-line is wsdl-validator, whilst ApacheDS remains the 3rd >> largest. >> > > We can also save around another 5MB by getting rid of the "same jar" being > copied under two locations (ex:- /lib, and /repository/components/plugins). > > Run the following command to find the number of duplicates: > > for f in `find -name '*.jar' | xargs ls -l | mawk '{print $5" "$8}' | sort > -gr | mawk '{gsub ("^.*/", "", $2); print $2}' | sort -u`; do echo "find > -name $f | head -n 1" > scr.out; sh scr.out; rm scr.out; done | wc -l > > a.out; for f in `find -name '*.jar' | xargs ls -l | mawk '{print $5" "$8}' | > sort -gr | mawk '{gsub ("^.*/", "", $2); print $2}' | sort -u`; do echo > "find -name $f" > scr.out; sh scr.out; rm scr.out; done | wc -l > b.out; > echo "(`cat b.out`-`cat a.out`)" | bc; rm a.out b.out > > And, the following command to find the saving in size (in MB): > > for f in `find -name '*.jar' | xargs ls -l | mawk '{print $5" "$8}' | sort > -gr | mawk '{gsub ("^.*/", "", $2); print $2}' | sort -u`; do echo "find > -name $f | head -n 1" > scr.out; sh scr.out; rm scr.out; done | xargs ls -l > | mawk '{print "+"$5}' | xargs echo "0"| bc > a.out; for f in `find -name > '*.jar' | xargs ls -l | mawk '{print $5" "$8}' | sort -gr | mawk '{gsub > ("^.*/", "", $2); print $2}' | sort -u`; do echo "find -name $f" > scr.out; > sh scr.out; rm scr.out; done | xargs ls -l | mawk '{print "+"$5}' | xargs > echo "0"| bc > b.out; echo "(`cat b.out`-`cat a.out`)/1024/1024" | bc; rm > a.out b.out > > Thanks, > Senaka. > > >> Thanks, >> Senaka. >> >> >>> which embeds icu, which is also available at >>> ./repository/components/plugins/com.ibm.icu-4.0.1.v20090415.jar. If this was >>> properly fixed, the reduction in size is 4.3MB. The fifth largest is Qpid, >>> which embeds derby (2.4MB), and mina twice (300K). If we switch to H2 and >>> get rid of the unwanted mina dependency, the gain is 2.7MB. >>> >>> So, as a summary, we could potentially save around 20MB of size just by >>> fixing the top 5 contributors to our binary size. >>> >>> Thanks, >>> Senaka. >>> >>> On Fri, Jun 10, 2011 at 9:14 PM, Anjana Fernando <[email protected]>wrote: >>> >>>> Hi Azeez, >>>> >>>> On Fri, Jun 10, 2011 at 8:28 PM, Afkham Azeez <[email protected]> wrote: >>>> >>>>> Can we make those optional features? >>>> >>>> >>>> Yeah, we should be able to, atleast the RDF support, it wont be that >>>> straightforward to do so, as in the way we typically install features, >>>> since >>>> it is directly attached to the data service feature functionality, but we >>>> can find a way, will do this for the next release. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> >>>>> >>>>> >>>>> On Fri, Jun 10, 2011 at 8:24 PM, Anjana Fernando <[email protected]>wrote: >>>>> >>>>>> Yeah, and our docs size have also grown in about 10MB. And we've new >>>>>> dependant libraries, such as, Jena, which is used for RDF handling = >>>>>> 7.3MB, >>>>>> and spreadsheet-ooxml-schemas = 6.4MB for Excel 2007 functionality. >>>>>> >>>>>> Cheers, >>>>>> Anjana. >>>>>> >>>>>> >>>>>> On Fri, Jun 10, 2011 at 4:43 PM, Nuwan Bandara <[email protected]>wrote: >>>>>> >>>>>>> Dashboard is optional in latest DS, so its not shipped by default. >>>>>>> however DS samples are quite big in size (~12MB) >>>>>>> >>>>>>> Regards, >>>>>>> /Nuwan >>>>>>> >>>>>>> >>>>>>> On Fri, Jun 10, 2011 at 4:32 PM, Afkham Azeez <[email protected]>wrote: >>>>>>> >>>>>>>> AS is 155 MB. DS has dashboard =15MB >>>>>>>> >>>>>>>> We have several jars which we can make optional or are duplicated. >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jun 10, 2011 at 3:53 PM, Samisa Abeysinghe <[email protected] >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> It is 170M, the largest. All others are less than 160M. Why is DS >>>>>>>>> so large? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Jun 10, 2011 at 5:25 AM, Sumedha Rubasinghe < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Jun 10, 2011 at 4:16 AM, Prabath Siriwardena < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Location : http://10.100.1.43/~carbon/releases/carbon/3.2.0/rc9/ >>>>>>>>>>> >>>>>>>>>>> P2 Repo : >>>>>>>>>>> http://10.100.1.43/~carbon/releases/carbon/3.2.0/rc9/p2-repo/ >>>>>>>>>>> >>>>>>>>>>> Public Location : >>>>>>>>>>> http://builder.wso2.org/~carbon/releases/carbon/3.2.0/rc9/[Please >>>>>>>>>>> give some time to complete copying] >>>>>>>>>>> >>>>>>>>>>> It seems copying to public location is still underway. >>>>>>>>>> /sumedha >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Revision : 107602 >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Thanks & Regards, >>>>>>>>>>> Prabath >>>>>>>>>>> >>>>>>>>>>> http://blog.facilelogin.com >>>>>>>>>>> http://RampartFAQ.com >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Carbon-dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Carbon-dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>> Samisa... >>>>>>>>> >>>>>>>>> Samisa Abeysinghe >>>>>>>>> VP Engineering >>>>>>>>> WSO2 Inc. >>>>>>>>> http://wso2.com >>>>>>>>> http://wso2.org >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Carbon-dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Afkham Azeez* >>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>> * <http://www.apache.org/>** >>>>>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>> twitter: >>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>> * >>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>> * >>>>>>>> * >>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Carbon-dev mailing list >>>>>>>> [email protected] >>>>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Thanks & Regards, >>>>>>> >>>>>>> Nuwan Bandara >>>>>>> Senior Software Engineer >>>>>>> >>>>>>> WSO2 Inc. | http://wso2.com >>>>>>> lean . enterprise . middleware >>>>>>> >>>>>>> http://nuwan.bandara.co >>>>>>> * >>>>>>> <http://www.nuwanbando.com/> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Carbon-dev mailing list >>>>>>> [email protected] >>>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Anjana Fernando* >>>>>> >>>>>> Senior Software Engineer >>>>>> WSO2 Inc. | http://wso2.com >>>>>> lean . enterprise . middleware >>>>>> >>>>>> _______________________________________________ >>>>>> Carbon-dev mailing list >>>>>> [email protected] >>>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Afkham Azeez* >>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>> * <http://www.apache.org/>** >>>>> email: **[email protected]* <[email protected]>* cell: +94 77 3320919 >>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>> twitter: >>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>> * >>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>> * >>>>> * >>>>> *Lean . Enterprise . Middleware* >>>>> >>>>> >>>>> _______________________________________________ >>>>> Carbon-dev mailing list >>>>> [email protected] >>>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Software Engineer >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>>> _______________________________________________ >>>> Carbon-dev mailing list >>>> [email protected] >>>> http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev >>>> >>>> >>> >>> >>> -- >>> *Senaka Fernando* >>> Product Manager - WSO2 Governance Registry; >>> Associate Technical Lead; WSO2 Inc.; http://wso2.com* >>> Member; Apache Software Foundation; http://apache.org >>> >>> E-mail: senaka AT wso2.com >>> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >>> Linked-In: http://linkedin.com/in/senakafernando >>> >>> *Lean . Enterprise . Middleware >>> >>> >> >> >> -- >> *Senaka Fernando* >> Product Manager - WSO2 Governance Registry; >> Associate Technical Lead; WSO2 Inc.; http://wso2.com* >> Member; Apache Software Foundation; http://apache.org >> >> E-mail: senaka AT wso2.com >> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 >> Linked-In: http://linkedin.com/in/senakafernando >> >> *Lean . Enterprise . Middleware >> >> > > > -- > *Senaka Fernando* > Product Manager - WSO2 Governance Registry; > Associate Technical Lead; WSO2 Inc.; http://wso2.com* > Member; Apache Software Foundation; http://apache.org > > E-mail: senaka AT wso2.com > **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818 > Linked-In: http://linkedin.com/in/senakafernando > > *Lean . Enterprise . Middleware > > > _______________________________________________ > Carbon-dev mailing list > [email protected] > http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev > > -- Sanjiva Weerawarana, Ph.D. Founder, Chairman & CEO; WSO2, Inc.; http://wso2.com/ email: [email protected]; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1 650 265 8311 blog: http://sanjiva.weerawarana.org/ Lean . Enterprise . Middleware
_______________________________________________ Carbon-dev mailing list [email protected] http://mail.wso2.org/cgi-bin/mailman/listinfo/carbon-dev
