Re: [RESULT] [VOTE] Release Apache James Mime4j 0.7 (second try)
On 30/07/11 21:27, Norman Maurer wrote: Thanks :) +1 Norman 2011/7/30 Stefano Bagnara: 2011/7/26 Eric Charles: Can someone update the sample page. I could do it, but it will be much faster and safer from mime4j specialists. https://svn.apache.org/repos/asf/james/mime4j/trunk/src/site/apt/usage.apt Thx. Done. The examples were already up to date as they were mainly related to the low level classes that didn't change much in 0.7. I updated the apt syntax so that generated links work. Stefano On 25/07/11 15:49, Eric Charles wrote: Norman has uploaded for the mirrors this morning. Still need some time to be updated. One small question : Mime4j (like in doc) or Mime4J (like in release notes) ? Thx. -- Eric Charles http://about.echarles.net - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org -- Eric Charles http://about.echarles.net - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: deploying james sites
On 30/07/11 13:24, Stefano Bagnara wrote:
2011/7/30 Eric Charles:
On 30/07/11 00:42, Stefano Bagnara wrote:
I guess it's an year or more I don't deploy james websites and I found
I don't know the updated way to do that.
I see the svn folder james/sites/trunk/www is outdated so I guess we
don't use it anymore (what about removing it?).
It is not used for now, but the goal is to recommit updated sites there, and
ask Apache Infra to SvnPubSub so we don't have to svn up and wait the sync
for now.
It's clear that we don't update that svn folder anymore. Infra doesn't
require anymore us to have the website in svn.
"mvn site-deploy" is much faster/easier than publishing to a local
folder and committing: expecially when you have to work with
multimodules sites (the staged site is not ok, and you have to
manually copy each module/target/site folder over the right svn
working copy in order to commit the stuff).
So unless anyone have a better workflow I propose to remove the svn
folder and simply use site-deploy.
mvn site-deploy is good to me.
Will you remove the www svn folder?
How am I supposed to update the web site?
I tried site:deploy for the project, but it doesn't deploy the full site
"mvn -Psite-reports site" creates the reports, but overwrite the index
from the previous command.
What are the right steps to deploy updated site and reports?
We talked about the reports some time ago and decided to have two separate
sites: the end user site and the site with reports.
I can't see the whole picture: where is deployed the end user site?
where is deployed the site with reports?
The goal was not to have the public site with some reports, even if it's
true that previous mime4j site had such reports.
So "the goal" is to remove reports from james.apache.org for all of
our product?? I don't like this goal.
I searched the archives and I don't find too much discussion/agreement
on something similar to this: I found some ideas, some plan, but
nothing like "ok, let's do this way, if anyone is against this speak
now" Have you any link for me to read?
I just reintroduced the reports for jDKIM as I think xrefs, coverage,
svn instructions and the other reports are really useful to the
developers coming to our site (I use them too). mvn site-deploy worked
fine for jDKIM.
I think you could copy some definitions from the site-reports profile to get
this reports in the public web site.
I think I found my way changing the main pom for jDKIM by removing
"inheritance" of the "generate reports variable" from the parent pom.
I'm also fine having the reports on the website.
How did you define it on jDKIM level ?
Do we still need the site-reports profile in [1] ?
Should we have this behaviour defined in [1] ?
[1] https://svn.apache.org/repos/asf/james/project/trunk/project/pom.xml
Also, I see the html generated from apt sources for mime4j don't
produce anymore valid html (bad links): is this something related to
newer maven site plugins? Do you know anything about this before I
start digging it?
For server, I remember I migrated the few apt to some xml (just to have a
uniform format).
I suppose the issue come from the new maven 3 site plugins.
No idea how to solve it. Eventually, you can migrate the apt to the xml.
I found some updated docs for apt.
Links to anchors are now {{{anchor}text}} and not {{{#anchor}text}}
Links to relative urls are now {{{./relative}text}} and not {{{relative}text}}.
The generated usage.html is good now (the sematic content was already
up-to-date with 0.7).
OK
Unfortunately mvn site-deploy for mime4j just failed because many
files inside the mime4j folder on people.apache.org are 644 instead of
664 so I get permission denied.
Many of them are "eric.apache" so I guess you (Eric) can fix them. (I
use a script I created some years ago to make sure permissions in www
for files owned by me are correct):
--
#!/bin/sh
find /www/james.apache.org ! -perm 775 -type d -user ${USER} -exec
chmod 775 {} \;
find /www/james.apache.org ! -perm 664 -type f -user ${USER} -exec
chmod 664 {} \;
find /www/james.apache.org -name \*\.cgi -type f -exec chmod 775 {} \;
-
The same happens with main project deployment. I've been able to
deploy some of the files by moving the "bad permissioned" files to an
"old" folder, but I can't do this for some of the bad permissioned
folders, like "js" and others (please remove "old" folder while you
fix the permissions, and maybe also remove ".tmp", "tmp").
Maybe we should also remove all of the .svn folders from there (as svn
is not updated anymore): they contains files with wrong permissions
owned by many apache devs (eric, norman, rdonking).
I faced same perm issue, and the workaround I applied was to copy to a
tmp folder, going to the server, and make a local copy.
I will start another thread about site deploy.
Stefano
-
To unsubscribe, e-mail: server-dev-unsub
Site deployment
Hi, We need to discuss the way we deploy web sites: 1. Via svn (commit in www project, and update on server). 2. Via svn (commit in www project, and automatically visible via svnpubsub). 3. Via scp (with file permissions issues...) 4. Via mvn site-deploy I understand there is a consensus for option 4 (mvn site-deploy). Can you confirm? Please also read for later evolutions: https://blogs.apache.org/infra/entry/the_asf_cms http://www.apache.org/dev/cms.html Thx. -- Eric Charles http://about.echarles.net - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Site deployment
2011/8/1 Eric Charles : > Hi, > > We need to discuss the way we deploy web sites: > > 1. Via svn (commit in www project, and update on server). > 2. Via svn (commit in www project, and automatically visible via svnpubsub). > 3. Via scp (with file permissions issues...) > 4. Via mvn site-deploy > > I understand there is a consensus for option 4 (mvn site-deploy). > > Can you confirm? Yes, #4 is my preferred solution, and in future we could even automate the site-deploy task from hudson. Also, site-deploy already takes care to update file permissions (this happens only when the site deploy task is completely successfull, so remember to take a look at the server perms manually if you can't successfully complete the site-deploy task) #1 and #2 are a waste of time for us and resources for ASF (svn space used by website updates is a lot and we don't need change tracking on that stuff). #3 is similar to #4 (#4 uses scp under the hood) but with less automation: I don't see many advantages in "mvn site && scp something" instead of "mvn site-deploy" (the main advantage of scp would be local review and selective copy, but I think we should try to avoid selective publishing) > Please also read for later evolutions: > https://blogs.apache.org/infra/entry/the_asf_cms > http://www.apache.org/dev/cms.html As long as we use maven sites we can ignore this. I didn't hear too much about this: which TLP already moved? I'm in favor of using a CMS for documentation instead of maven (or maybe a mix of the two) because I find maven site maintenaince is slow compared to a wiki or a web-based cms. I really don't like the fact that the "asf cms" needs SVN to update it (is this true?): then we would be stuck again to a development environment. I would very strongly prefer a real web based CMS so that we can update the website/docs whenever we have some minutes and some web access. Stefano - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Site deployment
Am 01.08.2011 10:28, schrieb Eric Charles: o discuss the way we deploy web sites: 1. Via svn (commit in www project, and update on server). 2. Via svn (commit in www project, and automatically visible via svnpubsub). 3. Via scp (with file permissions issues...) 4. Via mvn site-deploy I understand there is a consensus for option 4 (mvn site-deploy). Can you confirm? 4) +1 Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Site deployment
On Mon, Aug 1, 2011 at 10:18 AM, Norman Maurer wrote: > Am 01.08.2011 10:28, schrieb Eric Charles: >> >> o discuss the way we deploy web sites: >> >> 1. Via svn (commit in www project, and update on server). >> 2. Via svn (commit in www project, and automatically visible via >> svnpubsub). >> 3. Via scp (with file permissions issues...) >> 4. Via mvn site-deploy >> >> I understand there is a consensus for option 4 (mvn site-deploy). >> >> Can you confirm? > > 4) +1 I'd prefer 5) mvn site-deploy via svnpubsub but I don't think this is possible (yet) Robert - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Site deployment
2011/8/1 Robert Burrell Donkin : > On Mon, Aug 1, 2011 at 10:18 AM, Norman Maurer wrote: >> Am 01.08.2011 10:28, schrieb Eric Charles: >>> >>> o discuss the way we deploy web sites: >>> >>> 1. Via svn (commit in www project, and update on server). >>> 2. Via svn (commit in www project, and automatically visible via >>> svnpubsub). >>> 3. Via scp (with file permissions issues...) >>> 4. Via mvn site-deploy >>> >>> I understand there is a consensus for option 4 (mvn site-deploy). >>> >>> Can you confirm? >> >> 4) +1 > > I'd prefer 5) mvn site-deploy via svnpubsub but I don't think this is > possible (yet) Curiosity: what do you mean? a) automatically run "mvn site-deploy" whenever a commit is made in our svn sources b) change mvn site-deploy to be able to deploy to svn instead via scp and then use svnpubsub to export the svn content to people.a.o. Stefano - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
HBase message implementation
Hello, I'm having some issues putting things together when it comes to Message implementation. I will explain things right away. The current code [1], [2], and [3] is based on JPA implementation and it copies the message content into byte arrays. This does not seem to scale too well. There is also the issue of getting the info back from HBase. I agree with Eric and Norman that large content should be split and I'm thinking of providing a HBaseMessage implementation that should read data from HBase as demanded (ChunkedInputStream and ChunkedOutputStream as suggested by Norman [4] ). Do I have to copy the data when someone creates a new HBaseMessage, or as suggested by the streaming alternative, I can save a reference of SharedInputStream and when I save the message to the mailbox I can move the bytes to HBase. The way I see things is that HBaseMessage implementation is only used for retrieving data from HBase. It should not store the message body (in the constructor). Please, I need some clarification. Thanks, [1] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseMessage.java [2] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/AbstractHBaseMessage.java [3] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseStreamingMessage.java [4] https://github.com/rantav/hector/tree/master/core/src/main/java/me/prettyprint/cassandra/io -- Ioan Eugen Stan http://ieugen.blogspot.com/ - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: HBase message implementation
Hi Eugen, comments inside.. 2011/8/1 Ioan Eugen Stan : > Hello, > > I'm having some issues putting things together when it comes to > Message implementation. I will explain things right away. > The current code [1], [2], and [3] is based on JPA implementation and > it copies the message content into byte arrays. > This does not seem to scale too well. > > There is also the issue of getting the info back from HBase. I agree > with Eric and Norman that large content should be split and I'm > thinking of providing a HBaseMessage implementation that should read > data from HBase as demanded (ChunkedInputStream and > ChunkedOutputStream as suggested by Norman [4] ). > > Do I have to copy the data when someone creates a new HBaseMessage, or > as suggested by the streaming alternative, I can save a reference of > SharedInputStream and when I save the message to the mailbox I can > move the bytes to HBase. > > The way I see things is that HBaseMessage implementation is only used > for retrieving data from HBase. It should not store the message body > (in the constructor). Exactly.. The only special case is the MessageMapper.copy(..) but maybe thiscan be handled in a more efficient way in hbase... > > Please, I need some clarification. > > Thanks, > > [1] > http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseMessage.java > [2] > http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/AbstractHBaseMessage.java > [3] > http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseStreamingMessage.java > [4] > https://github.com/rantav/hector/tree/master/core/src/main/java/me/prettyprint/cassandra/io > > -- > Ioan Eugen Stan > http://ieugen.blogspot.com/ Bye, Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: HBase message implementation
On 01/08/11 18:52, Norman Maurer wrote: Hi Eugen, comments inside.. 2011/8/1 Ioan Eugen Stan: Hello, I'm having some issues putting things together when it comes to Message implementation. I will explain things right away. The current code [1], [2], and [3] is based on JPA implementation and it copies the message content into byte arrays. This does not seem to scale too well. There is also the issue of getting the info back from HBase. I agree with Eric and Norman that large content should be split and I'm thinking of providing a HBaseMessage implementation that should read data from HBase as demanded (ChunkedInputStream and ChunkedOutputStream as suggested by Norman [4] ). Do I have to copy the data when someone creates a new HBaseMessage, or as suggested by the streaming alternative, I can save a reference of SharedInputStream and when I save the message to the mailbox I can move the bytes to HBase. The way I see things is that HBaseMessage implementation is only used for retrieving data from HBase. It should not store the message body (in the constructor). Exactly.. The only special case is the MessageMapper.copy(..) but maybe thiscan be handled in a more efficient way in hbase... The byte loading into memory can be further deferred from the HBaseMessage constructor to a HBaseMessageMapper method, but at the end, the memory will be used by the bytes because there is no native streaming support on hbase side. To mitigate the effect, you can implement ChunkedInput/OuputStream that will split the stream in more limited chunks, saving some peak memory usage. The call to these Chunk classes that transforms the stream to the bytes is probably best placed in the HBaseMessageMapper to avoid pollute the HBaseMessage constructor with many byte arrays, also loosing the positive effect of the chunks. You will need to loop and instantiate one HBase Put per chunk in the HBaseMessageMapper. Please, I need some clarification. Thanks, [1] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseMessage.java [2] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/AbstractHBaseMessage.java [3] http://code.google.com/a/apache-extras.org/p/mailbox-hdfs/source/browse/src/main/java/org/apache/james/mailbox/hbase/mail/model/hbase/HBaseStreamingMessage.java [4] https://github.com/rantav/hector/tree/master/core/src/main/java/me/prettyprint/cassandra/io -- Ioan Eugen Stan http://ieugen.blogspot.com/ Bye, Norman - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org -- Eric Charles http://about.echarles.net - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
