Author: sebb
Date: Fri May 9 14:37:13 2008
New Revision: 654944
URL: http://svn.apache.org/viewvc?rev=654944&view=rev
Log:
Bug 44418/42178 - CSV Dataset file handling improvements
Modified:
jakarta/jmeter/trunk/docs/changes.html
jakarta/jmeter/trunk/docs/images/screenshots/csvdatasetconfig.png
jakarta/jmeter/trunk/docs/usermanual/component_reference.html
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSet.java
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetBeanInfo.java
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetResources.properties
jakarta/jmeter/trunk/xdocs/changes.xml
jakarta/jmeter/trunk/xdocs/images/screenshots/csvdatasetconfig.png
jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml
Modified: jakarta/jmeter/trunk/docs/changes.html
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/docs/changes.html?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
--- jakarta/jmeter/trunk/docs/changes.html (original)
+++ jakarta/jmeter/trunk/docs/changes.html Fri May 9 14:37:13 2008
@@ -177,6 +177,12 @@
</p>
<p >
+
+The CSV Dataset configuration element has new file sharing options: per thread
group, per thread, per identifier.
+This allows for more flexible file processing, e.g. each thread can process
the same data in the same order.
+
+ </p>
+
<p >
Switch
Controller now works properly with functions and variables,
and the condition can now be a name instead of a number.
Simple Controller now works properly under a While Controller
@@ -652,6 +658,11 @@
</li>
+
<li >
+ Bug 44418/42178
- CSV Dataset file handling improvements
+ </li>
+
+
</ul>
<h4 >
Non-functional
changes
Modified: jakarta/jmeter/trunk/docs/images/screenshots/csvdatasetconfig.png
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/docs/images/screenshots/csvdatasetconfig.png?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
Binary files - no diff available.
Modified: jakarta/jmeter/trunk/docs/usermanual/component_reference.html
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/docs/usermanual/component_reference.html?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
--- jakarta/jmeter/trunk/docs/usermanual/component_reference.html (original)
+++ jakarta/jmeter/trunk/docs/usermanual/component_reference.html Fri May 9
14:37:13 2008
@@ -6725,15 +6725,18 @@
<p >
- The file is only opened once, and each thread will use a different line
from the file.
- Lines are read as the threads need them.
+ By default, the file is only opened once, and each thread will use a
different line from the file.
+ See the description of the Share mode below for additional options
(JMeter 2.3.2+).
+ Lines are read at the start of each test iteration.
+ The file name and mode are resolved in the first iteration.
</p>
<p><table border="1" bgcolor="#bbbb00" width="50%"
cellspacing="0" cellpadding="2">
- <tr><td> CSV
Dataset variables are defined after configuration processing is completed,
- so they cannot be used for some configuration items - such as JDBC
Config -
+ <tr><td> CSV
Dataset variables are defined at the start of each test iteration.
+ As this is after configuration processing is completed,
+ they cannot be used for some configuration items - such as JDBC Config
-
that process their contents at configuration time (see
<a
href="http://issues.apache.org/bugzilla/show_bug.cgi?id=40934";>
Bug 40394
@@ -6783,7 +6786,7 @@
<p><b>Control Panel</b></p>
- <div align="center"><img
width='308' height='278' src="../images/screenshots/csvdatasetconfig.png"></div>
+ <div align="center"><img
width='396' height='301' src="../images/screenshots/csvdatasetconfig.png"></div>
<p>
<b>Parameters</b>
<table border="1" cellspacing="0" cellpadding="2">
@@ -6806,6 +6809,9 @@
Absolute file names are also supported, but note that they are unlikely to
work in remote mode,
unless the remote server has the same directory structure.
+ If the same physical file is referenced in two different ways - e.g.
csvdata.txt and ./csvdata.txt -
+ then these are treated as different files.
+ If the OS does not distinguish between upper and lower case, csvData.TXT
would also be opened separately.
</td>
<td>
@@ -6862,6 +6868,45 @@
Yes
</td>
</tr>
+ <tr>
+ <td>Sharing mode</td>
+ <td>
+
+
<ul >
+
+
+
<li >
+ All threads -
(the default) the file is shared between all the threads.
+ </li>
+
+
+
<li >
+ Current thread
group - each file is opened once for each thread group in which the element
appears
+ </li>
+
+
+
<li >
+ Current thread
- each file is opened separately for each thread
+ </li>
+
+
+
<li >
+ Identifier -
all threads sharing the same identifier share the same file.
+ So for example if you have 4 thread groups, you could use a common id for
two or more of the groups
+ to share the file between them.
+ Or you could use the thread number to share the file between the same thread
numbers in different thread groups.
+
+ </li>
+
+
+ </ul>
+
+
+ </td>
+ <td>
+
Yes
+ </td>
+ </tr>
</table>
</p>
</td></tr>
Modified:
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSet.java
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSet.java?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
---
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSet.java
(original)
+++
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSet.java
Fri May 9 14:37:13 2008
@@ -28,6 +28,7 @@
import org.apache.jmeter.save.CSVSaveService;
import org.apache.jmeter.services.FileServer;
import org.apache.jmeter.testbeans.TestBean;
+import org.apache.jmeter.threads.JMeterContext;
import org.apache.jmeter.threads.JMeterVariables;
import org.apache.jmeter.util.JMeterUtils;
import org.apache.jorphan.logging.LoggingManager;
@@ -35,10 +36,35 @@
import org.apache.jorphan.util.JOrphanUtils;
import org.apache.log.Logger;
+/**
+ * Read lines from a file and split int variables.
+ *
+ * The iterationStart() method is used to set up each set of values.
+ *
+ * By default, the same file is shared between all threads
+ * (and other thread groups, if they use the same file name).
+ *
+ * The shareMode can be set to:
+ * <ul>
+ * <li>All threads - default, as described above</li>
+ * <li>Current thread group</li>
+ * <li>Current thread</li>
+ * <li>Identifier - all threads sharing the same identifier</li>
+ * </ul>
+ *
+ * The class uses the FileServer alias mechanism to provide the different
share modes.
+ * For all threads, the file alias is set to the file name.
+ * Otherwise, a suffix is appended to the filename to make it unique within
the required context.
+ * For current thread group, the thread group identityHashcode is used;
+ * for individual threads, the thread hashcode is used as the suffix.
+ * Or the user can provide their own suffix, in which case the file is shared
between all
+ * threads with the same suffix.
+ *
+ */
public class CSVDataSet extends ConfigTestElement implements TestBean,
LoopIterationListener {
private static final Logger log = LoggingManager.getLoggerForClass();
- private static final long serialVersionUID = 2;
+ private static final long serialVersionUID = 232L;
private static final String EOFVALUE = // value to return at EOF
JMeterUtils.getPropDefault("csvdataset.eofstring", "<EOF>");
//$NON-NLS-1$ //$NON-NLS-2$
@@ -51,13 +77,17 @@
private transient String delimiter;
- private transient boolean quoted = false;
+ private transient boolean quoted;
private transient boolean recycle = true;
- private transient boolean stopThread = false;
+ private transient boolean stopThread;
+
+ private transient String[] vars;
+
+ private transient String alias;
- transient private String[] vars;
+ private transient String shareMode;
private Object readResolve(){
recycle = true;
@@ -70,9 +100,26 @@
*/
public void iterationStart(LoopIterationEvent iterEvent) {
FileServer server = FileServer.getFileServer();
- String _fileName = getFilename();
+ final JMeterContext context = getThreadContext();
if (vars == null) {
- server.reserveFile(_fileName, getFileEncoding());
+ String _fileName = getFilename();
+ String mode = getShareMode();
+ int modeInt = CSVDataSetBeanInfo.getShareModeAsInt(mode);
+ switch(modeInt){
+ case CSVDataSetBeanInfo.SHARE_ALL:
+ alias = _fileName;
+ break;
+ case CSVDataSetBeanInfo.SHARE_GROUP:
+ alias =
_fileName+"@"+System.identityHashCode(context.getThreadGroup());
+ break;
+ case CSVDataSetBeanInfo.SHARE_THREAD:
+ alias =
_fileName+"@"+System.identityHashCode(context.getThread());
+ break;
+ default:
+ alias = _fileName+"@"+mode; // user-specified key
+ break;
+ }
+ server.reserveFile(_fileName, getFileEncoding(), alias);
vars = JOrphanUtils.split(getVariableNames(), ","); //
$NON-NLS-1$
}
try {
@@ -80,8 +127,9 @@
if (delim.equals("\\t")) { // $NON-NLS-1$
delim = "\t";// Make it easier to enter a Tab
// $NON-NLS-1$
}
- JMeterVariables threadVars =
this.getThreadContext().getVariables();
- String line = server.readLine(_fileName,getRecycle());
+ // TODO: fetch this once as per vars above?
+ JMeterVariables threadVars = context.getVariables();
+ String line = server.readLine(alias,getRecycle());
if (line!=null) {// i.e. not EOF
String[] lineValues = getQuotedData() ?
CSVSaveService.csvReadFile(new BufferedReader(new
StringReader(line)), delim.charAt(0))
@@ -180,5 +228,13 @@
public void setStopThread(boolean value) {
this.stopThread = value;
}
+
+ public String getShareMode() {
+ return shareMode;
+ }
+
+ public void setShareMode(String value) {
+ this.shareMode = value;
+ }
}
Modified:
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetBeanInfo.java
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetBeanInfo.java?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
---
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetBeanInfo.java
(original)
+++
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetBeanInfo.java
Fri May 9 14:37:13 2008
@@ -19,13 +19,10 @@
package org.apache.jmeter.config;
import java.beans.PropertyDescriptor;
+import java.util.ResourceBundle;
import org.apache.jmeter.testbeans.BeanInfoSupport;
-/**
- * @author mstover
- *
- */
public class CSVDataSetBeanInfo extends BeanInfoSupport {
// These names must agree case-wise with the variable and property names
@@ -36,11 +33,25 @@
private static final String RECYCLE = "recycle";
//$NON-NLS-1$
private static final String STOPTHREAD = "stopThread";
//$NON-NLS-1$
private static final String QUOTED_DATA = "quotedData";
//$NON-NLS-1$
-
+ private static final String SHAREMODE = "shareMode";
//$NON-NLS-1$
+
+ private static final String[] SHARE_TAGS = new String[3];
+ static final int SHARE_ALL = 0;
+ static final int SHARE_GROUP = 1;
+ static final int SHARE_THREAD = 2;
+
+
public CSVDataSetBeanInfo() {
super(CSVDataSet.class);
+
+ ResourceBundle rb = (ResourceBundle)
getBeanDescriptor().getValue(RESOURCE_BUNDLE);
+// These must agree with the resources
+ SHARE_TAGS[SHARE_ALL] = rb.getString("shareMode.all");
//$NON-NLS-1$
+ SHARE_TAGS[SHARE_GROUP] = rb.getString("shareMode.group");
//$NON-NLS-1$
+ SHARE_TAGS[SHARE_THREAD] = rb.getString("shareMode.thread");
//$NON-NLS-1$
+
createPropertyGroup("csv_data", //$NON-NLS-1$
- new String[] { FILENAME, FILE_ENCODING, VARIABLE_NAMES,
DELIMITER, QUOTED_DATA, RECYCLE, STOPTHREAD });
+ new String[] { FILENAME, FILE_ENCODING, VARIABLE_NAMES,
DELIMITER, QUOTED_DATA, RECYCLE, STOPTHREAD, SHAREMODE });
PropertyDescriptor p = property(FILENAME);
p.setValue(NOT_UNDEFINED, Boolean.TRUE);
@@ -73,5 +84,25 @@
p = property(STOPTHREAD);
p.setValue(NOT_UNDEFINED, Boolean.TRUE);
p.setValue(DEFAULT, Boolean.FALSE);
+
+ p = property(SHAREMODE); //$NON-NLS-1$
+ p.setValue(NOT_UNDEFINED, Boolean.TRUE);
+ p.setValue(DEFAULT, SHARE_TAGS[0]);
+ p.setValue(NOT_OTHER, Boolean.FALSE);
+ p.setValue(NOT_EXPRESSION, Boolean.FALSE);
+ p.setValue(TAGS, SHARE_TAGS);
}
+
+ // TODO need to find better way to do this
+ public static int getShareModeAsInt(String mode) {
+ if (mode.length() == 0){
+ return SHARE_ALL; // default (e.g. if test plan does not have
definition)
+ }
+ for (int i = 0; i < SHARE_TAGS.length; i++) {
+ if (SHARE_TAGS[i].equals(mode)) {
+ return i;
+ }
+ }
+ return -1;
+ }
}
Modified:
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetResources.properties
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetResources.properties?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
---
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetResources.properties
(original)
+++
jakarta/jmeter/trunk/src/components/org/apache/jmeter/config/CSVDataSetResources.properties
Fri May 9 14:37:13 2008
@@ -13,4 +13,9 @@
recycle.displayName=Recycle on EOF ?
recycle.shortDescription=Should the file be re-read from the start on reaching
EOF ?
stopThread.displayName=Stop thread on EOF ?
-stopThread.shortDescription=Should the thread be stopped on reaching EOF (if
Recycle is false) ?
\ No newline at end of file
+stopThread.shortDescription=Should the thread be stopped on reaching EOF (if
Recycle is false) ?
+shareMode.displayName=Sharing mode
+shareMode.shortDescription=Select which threads share the same file pointer
+shareMode.all=All threads
+shareMode.group=Current thread group
+shareMode.thread=Current thread
Modified: jakarta/jmeter/trunk/xdocs/changes.xml
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/xdocs/changes.xml?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
--- jakarta/jmeter/trunk/xdocs/changes.xml (original)
+++ jakarta/jmeter/trunk/xdocs/changes.xml Fri May 9 14:37:13 2008
@@ -50,6 +50,11 @@
This bug has been corrected.
</p>
+<p>
+The CSV Dataset configuration element has new file sharing options: per thread
group, per thread, per identifier.
+This allows for more flexible file processing, e.g. each thread can process
the same data in the same order.
+</p>
+
<p>Switch Controller now works properly with functions and variables,
and the condition can now be a name instead of a number.
Simple Controller now works properly under a While Controller</p>
@@ -193,6 +198,7 @@
<li>Mirror server can now be run independently (mirror-server.cmd and
mirror-server.sh)</li>
<li>Bug 19128 - Added multiple file POST support to HTTP Samplers</li>
<li>Allow use of special name LAST to mean the last test run; applies to -t,
-l, -j flags</li>
+<li>Bug 44418/42178 - CSV Dataset file handling improvements</li>
</ul>
<h4>Non-functional changes</h4>
Modified: jakarta/jmeter/trunk/xdocs/images/screenshots/csvdatasetconfig.png
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/xdocs/images/screenshots/csvdatasetconfig.png?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
Binary files - no diff available.
Modified: jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml
URL:
http://svn.apache.org/viewvc/jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml?rev=654944&r1=654943&r2=654944&view=diff
==============================================================================
--- jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml (original)
+++ jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml Fri May 9
14:37:13 2008
@@ -2148,7 +2148,7 @@
<br></br>
</description>
-<component name="CSV Data Set Config" index="§-num;.4.1" width="308"
height="278" screenshot="csvdatasetconfig.png">
+<component name="CSV Data Set Config" index="§-num;.4.1" width="396"
height="301" screenshot="csvdatasetconfig.png">
<description>
<p>
CSV Data Set Config is used to read lines from a file, and split them
into variables.
@@ -2157,11 +2157,14 @@
Previously it was necessary to choose a delimiter that was not used in
any values.
</p>
<p>
- The file is only opened once, and each thread will use a different line
from the file.
- Lines are read as the threads need them.
+ By default, the file is only opened once, and each thread will use a
different line from the file.
+ See the description of the Share mode below for additional options
(JMeter 2.3.2+).
+ Lines are read at the start of each test iteration.
+ The file name and mode are resolved in the first iteration.
</p>
- <note>CSV Dataset variables are defined after configuration processing
is completed,
- so they cannot be used for some configuration items - such as JDBC
Config -
+ <note>CSV Dataset variables are defined at the start of each test
iteration.
+ As this is after configuration processing is completed,
+ they cannot be used for some configuration items - such as JDBC Config
-
that process their contents at configuration time (see <a
href="http://issues.apache.org/bugzilla/show_bug.cgi?id=40934";>Bug 40394 </a>)
However the variables do work in the HTTP Auth Manager, as the username
etc are processed at run-time.
</note>
@@ -2185,6 +2188,9 @@
<b>Relative file names are resolved with respect to the path of the active
test plan.</b>
Absolute file names are also supported, but note that they are unlikely to
work in remote mode,
unless the remote server has the same directory structure.
+ If the same physical file is referenced in two different ways - e.g.
csvdata.txt and ./csvdata.txt -
+ then these are treated as different files.
+ If the OS does not distinguish between upper and lower case, csvData.TXT
would also be opened separately.
</property>
<property name="File Encoding" required="No">The encoding to be used to read
the file, if not the platform default.</property>
<property name="Variable Names" required="Yes">List of variable names
(comma-delimited)</property>
@@ -2194,6 +2200,18 @@
<property name="Allow quoted data?" required="Yes">Should the CSV file allow
values to be quoted?</property>
<property name="Recycle on EOF?" required="Yes">Should the file be re-read
from the beginning on reaching EOF? (default is true)</property>
<property name="Stop thread on EOF?" required="Yes">Should the thread be
stopped on EOF, if Recycle is false? (default is false)</property>
+ <property name="Sharing mode" required="Yes">
+ <ul>
+ <li>All threads - (the default) the file is shared between all the
threads.</li>
+ <li>Current thread group - each file is opened once for each thread group in
which the element appears</li>
+ <li>Current thread - each file is opened separately for each thread</li>
+ <li>Identifier - all threads sharing the same identifier share the same file.
+ So for example if you have 4 thread groups, you could use a common id for
two or more of the groups
+ to share the file between them.
+ Or you could use the thread number to share the file between the same thread
numbers in different thread groups.
+ </li>
+ </ul>
+ </property>
</properties>
</component>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
