[GitHub] activemq-artemis pull request: Large message database
Github user clebertsuconic commented on the pull request: https://github.com/apache/activemq-artemis/pull/502#issuecomment-217053349 @mtaylor I squashed 152e572 into d19a7dd... I preferred using an intermediate branch myself instead of asking you.. I was pretty sure about being the right thing to do.. let me explain: Basically d19a7dd is breaking the testsuite, and 152e572 is fixing it. Well... you would have a commit with a broken code, and I would avoid that if possible... these committs belonged together so I squahsed them right before pushing upstream. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Large message database
Github user mtaylor commented on the pull request: https://github.com/apache/activemq-artemis/pull/502#issuecomment-217010719 @clebertsuconic What happened to the last commit? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Adding support for password masking to the Artemis
That sounds fine. I would expect the PR to also include a test or two (as mentioned previously) plus documentation updates as well. A full blown example (e.g. based on the "security" example) would be nice to have as well. Justin - Original Message - From: "adclayton" To: dev@activemq.apache.org Sent: Wednesday, May 4, 2016 3:09:35 PM Subject: Re: Adding support for password masking to the Artemis OK well what I am thinking about is basically adding a nullable property to the PropertiesLoginModule that will allow the user to provide the classpath to an implementation of the org.apache.activemq.artemis.utils.SensitiveDataCodec, so that essentially the login.config would look like the following. activemq { com.paypal.credit.apache.artemis.passwordmasking.CryptoMonsterLogin required debug=false org.apache.activemq.jaas.properties.decoder="com.foo.FooDecoder;key=value" org.apache.activemq.jaas.properties.user="artemis-users.properties" org.apache.activemq.jaas.properties.role="artemis-roles.properties"; }; This approach should allow users to configure password marking using the same interface from the previous version. Also I believe we can allow masked and unmasked passwords to live in the same files by simply requiring the masked passwords to be encased with ENC(). An example of how the users.properties file would look is shown below. unmaskedpassword=unmaskedpassword maskedpassword=ENC(fafafsdgaqgegsagegasge) How does this sound? -- View this message in context: http://activemq.2283324.n4.nabble.com/Adding-support-for-password-masking-to-the-Artemis-tp4711531p4711571.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
[GitHub] activemq-artemis pull request: More test fixes
Github user asfgit closed the pull request at: https://github.com/apache/activemq-artemis/pull/505 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Adding support for password masking to the Artemis
OK well what I am thinking about is basically adding a nullable property to the PropertiesLoginModule that will allow the user to provide the classpath to an implementation of the org.apache.activemq.artemis.utils.SensitiveDataCodec, so that essentially the login.config would look like the following. activemq { com.paypal.credit.apache.artemis.passwordmasking.CryptoMonsterLogin required debug=false org.apache.activemq.jaas.properties.decoder="com.foo.FooDecoder;key=value" org.apache.activemq.jaas.properties.user="artemis-users.properties" org.apache.activemq.jaas.properties.role="artemis-roles.properties"; }; This approach should allow users to configure password marking using the same interface from the previous version. Also I believe we can allow masked and unmasked passwords to live in the same files by simply requiring the masked passwords to be encased with ENC(). An example of how the users.properties file would look is shown below. unmaskedpassword=unmaskedpassword maskedpassword=ENC(fafafsdgaqgegsagegasge) How does this sound? -- View this message in context: http://activemq.2283324.n4.nabble.com/Adding-support-for-password-masking-to-the-Artemis-tp4711531p4711571.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
[GitHub] activemq-artemis pull request: More test fixes
GitHub user jbertram opened a pull request: https://github.com/apache/activemq-artemis/pull/505 More test fixes You can merge this pull request into a Git repository by running: $ git pull https://github.com/jbertram/activemq-artemis master_work Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/505.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #505 commit 170d5c12966173be71c9251e2a80ffb777025145 Author: jbertram Date: 2016-05-04T19:12:02Z Fix ActiveMQServerControlUsingCoreTest When using the "core" control some of the metrics change slightly and those changes have to be accounted for in the tests. commit d36a22c4a7185a08ed5887e30705c2b263446892 Author: jbertram Date: 2016-05-04T20:14:32Z Fix AutoGroupingTest failure commit 89a0ffe1b2df50167af98248a7837aeb1f3b3999 Author: jbertram Date: 2016-05-04T20:16:51Z Fix GroupIDTest failure commit 500a734d047a3fb6445eff0fbb80c6b23593b21c Author: jbertram Date: 2016-05-04T20:26:59Z Fix AutoDeleteJmsQueueTest commit 1d770e49f00685e6da69c996928624981a9a740a Author: jbertram Date: 2016-05-04T20:28:23Z Fix RaceOnSyncLargeMessageOverReplicationTest --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Switch File Based Large Message Tes...
Github user asfgit closed the pull request at: https://github.com/apache/activemq-artemis/pull/504 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Test-suite fixes
Github user asfgit closed the pull request at: https://github.com/apache/activemq-artemis/pull/503 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Test-suite fixes
Github user clebertsuconic commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/503#discussion_r62092627 --- Diff: tests/extra-tests/src/test/java/org/apache/activemq/artemis/tests/extras/byteman/LargeMessageOverReplicationTest.java --- @@ -29,6 +29,7 @@ import org.apache.activemq.artemis.core.config.Configuration; import org.apache.activemq.artemis.core.config.CoreQueueConfiguration; import org.apache.activemq.artemis.core.server.ActiveMQServer; +import org.apache.activemq.artemis.core.settings.impl.AddressSettings; --- End diff -- accident? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Switch File Based Large Message Tes...
GitHub user mtaylor opened a pull request: https://github.com/apache/activemq-artemis/pull/504 Switch File Based Large Message Tests You can merge this pull request into a Git repository by running: $ git pull https://github.com/mtaylor/activemq-artemis FileLargeMessageTests Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/504.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #504 commit b5c769a645163e5c7c998276b47f01fee0b2a44a Author: Martyn Taylor Date: 2016-05-04T18:28:06Z Switch File Based Large Message Tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: ARTEMIS-512 fix variable name confl...
GitHub user jbertram opened a pull request: https://github.com/apache/activemq-artemis/pull/503 ARTEMIS-512 fix variable name conflict You can merge this pull request into a Git repository by running: $ git pull https://github.com/jbertram/activemq-artemis master_work Alternatively you can review and apply these changes as the patch at: https://github.com/apache/activemq-artemis/pull/503.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #503 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] activemq-artemis pull request: Large message database
Github user asfgit closed the pull request at: https://github.com/apache/activemq-artemis/pull/502 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Embedded REST - Missing msg-pull-subscriptions when request HEAD
What documentation are you reading? The documentation at http://activemq.apache.org/artemis/docs/1.2.0/rest.html indicates that you will receive this exact kind of response when you send a HEAD request to a queue. The "msg-pull-consumers" and "msg-push-consumers" are what you need to create a consumer on a queue as noted in the documentation under the "Queue Resource Response Headers" heading. This process is also outlined in the documentation under the "Creating an Auto-Ack Consumer or Subscription" heading. I believe you are getting confused between a queue and a topic. Inspect the documentation more closely and I think you'll find the information you need. Justin - Original Message - From: "Julien d" To: dev@activemq.apache.org Sent: Wednesday, May 4, 2016 10:30:48 AM Subject: Re: Embedded REST - Missing msg-pull-subscriptions when request HEAD Hi, I am facing an unexpected issue. As I understand from the documentation I have to register consumers subscribing. However using curl, how could I do that? At the moment when I issue curl --head http://localhost:8081/queues/orders, I obtain as response : HTTP/1.1 200 Ok connection: keep-alive msg-create-with-id: http://localhost:8081/queues/orders/create/{id} date: Wed, 04 May 2016 11:38:58 GMT server: D. Rogatkin's TJWS based on Acme.Serve/Version 1.70, $Revision: 1.194 $ msg-pull-consumers: http://localhost:8081/queues/orders/pull-consumers msg-create: http://localhost:8081/queues/orders/create msg-push-consumers: http://localhost:8081/queues/orders/push-consumers keep-alive: timeout=30, max=100 mime-version: 1.0 URLs mentioned in the documentation : |msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions| are not present. I want to be able to receive messages sent to a queue from cURL. How can I have the needed URL path enabled and functional in my case for cURL in my case? Best regards, Julien D. Alternative email address : julie...@protonmail.ch On 05/03/2016 11:55 PM, Justin Bertram wrote: > One important thing to realize is that the body of a message is just a byte > array. Try this: > > byte[] body = new byte[message2.getBodySize()]; > msg.getBodyBuffer().readBytes(body); > System.out.println(new String(body)); > > > Justin > > - Original Message - > From: "Julien d" > To: dev@activemq.apache.org > Sent: Tuesday, May 3, 2016 2:46:53 PM > Subject: Re: Embedded REST - monitor queues (without JMS) > > Thanks, now it works much better however I do not get what I expected as > output. > > ClientSession session = clientSessionFactory.createSession(); > session.start(); ClientConsumer consumer = session.createConsumer("orders"); > ClientMessage message2 = consumer.receive(); > System.out.println(message2.getBodyBuffer().toString()); consumer.close(); > session.close(); > > is my current code I get as output following my request : > > > org.apache.activemq.artemis.core.buffers.impl.ResetLimitWrappedActiveMQBuffer@45385f75 > > > And if I do : > > System.out.println(message2.getBodyBuffer().readString()); > > I get this error : > > [Tue May 03 22:40:44 EEST 2016] Servlet for path '/' already defined > and no default will be used. Exception in thread "main" > java.lang.OutOfMemoryError: Java heap space at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readSimpleStringInternal(ChannelBufferWrapper.java:89) > at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readStringInternal(ChannelBufferWrapper.java:113) > at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readString(ChannelBufferWrapper.java:96) > at > > org.apache.activemq.artemis.jms.example.EmbeddedRest.main(EmbeddedRest.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) > > I am sending this : BilliPhone > 4$199.99 through : curl -X POST > -d...@test.xml http://localhost:8081/queues/orders/create > BilliPhone 4$199.99 > Why would not it display the message content? > > Julien D. > > Alternative email address : julie...@protonmail.ch > > On 05/03/2016 10:27 PM, Justin Bertram wrote: >>> Did I do anything wrong? >> Yes. :) >> >> First, you aren't protecting against an NPE in the case where >> consumer.receive(1000) times out and returns null. >> >> Second, you aren't calling start() on your session so there's no possibility >> of you receiving mess
Re: Embedded REST - Missing msg-pull-subscriptions when request HEAD
Hi, I am facing an unexpected issue. As I understand from the documentation I have to register consumers subscribing. However using curl, how could I do that? At the moment when I issue curl --head http://localhost:8081/queues/orders, I obtain as response : HTTP/1.1 200 Ok connection: keep-alive msg-create-with-id: http://localhost:8081/queues/orders/create/{id} date: Wed, 04 May 2016 11:38:58 GMT server: D. Rogatkin's TJWS based on Acme.Serve/Version 1.70, $Revision: 1.194 $ msg-pull-consumers: http://localhost:8081/queues/orders/pull-consumers msg-create: http://localhost:8081/queues/orders/create msg-push-consumers: http://localhost:8081/queues/orders/push-consumers keep-alive: timeout=30, max=100 mime-version: 1.0 URLs mentioned in the documentation : |msg-pull-subscriptions: http://example.com/topics/jms.topic.foo/pull-subscriptions msg-push-subscriptions: http://example.com/topics/jms.topic.foo/push-subscriptions| are not present. I want to be able to receive messages sent to a queue from cURL. How can I have the needed URL path enabled and functional in my case for cURL in my case? Best regards, Julien D. Alternative email address : julie...@protonmail.ch On 05/03/2016 11:55 PM, Justin Bertram wrote: > One important thing to realize is that the body of a message is just a byte > array. Try this: > > byte[] body = new byte[message2.getBodySize()]; > msg.getBodyBuffer().readBytes(body); > System.out.println(new String(body)); > > > Justin > > - Original Message - > From: "Julien d" > To: dev@activemq.apache.org > Sent: Tuesday, May 3, 2016 2:46:53 PM > Subject: Re: Embedded REST - monitor queues (without JMS) > > Thanks, now it works much better however I do not get what I expected as > output. > > ClientSession session = clientSessionFactory.createSession(); > session.start(); ClientConsumer consumer = session.createConsumer("orders"); > ClientMessage message2 = consumer.receive(); > System.out.println(message2.getBodyBuffer().toString()); consumer.close(); > session.close(); > > is my current code I get as output following my request : > > > org.apache.activemq.artemis.core.buffers.impl.ResetLimitWrappedActiveMQBuffer@45385f75 > > > And if I do : > > System.out.println(message2.getBodyBuffer().readString()); > > I get this error : > > [Tue May 03 22:40:44 EEST 2016] Servlet for path '/' already defined > and no default will be used. Exception in thread "main" > java.lang.OutOfMemoryError: Java heap space at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readSimpleStringInternal(ChannelBufferWrapper.java:89) > at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readStringInternal(ChannelBufferWrapper.java:113) > at > > org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readString(ChannelBufferWrapper.java:96) > at > > org.apache.activemq.artemis.jms.example.EmbeddedRest.main(EmbeddedRest.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) > > I am sending this : BilliPhone > 4$199.99 through : curl -X POST > -d...@test.xml http://localhost:8081/queues/orders/create > BilliPhone 4$199.99 > Why would not it display the message content? > > Julien D. > > Alternative email address : julie...@protonmail.ch > > On 05/03/2016 10:27 PM, Justin Bertram wrote: >>> Did I do anything wrong? >> Yes. :) >> >> First, you aren't protecting against an NPE in the case where >> consumer.receive(1000) times out and returns null. >> >> Second, you aren't calling start() on your session so there's no possibility >> of you receiving messages. >> >> >> Justin >> >> - Original Message - >> From: "Julien d" >> To: dev@activemq.apache.org >> Sent: Tuesday, May 3, 2016 2:15:44 PM >> Subject: Re: Embedded REST - monitor queues (without JMS) >> >> Hi again, >> >> Using the consumer method (which is best for my case) I run into a >> NullPointer exception when accessing my message body : >> >> [Tue May 03 22:10:25 EEST 2016] Servlet for path '/' already defined >> and no default will be used. >> Exception in thread "main" java.lang.NullPointerException >> at >> >> org.apache.activemq.artemis.jms.example.EmbeddedRest.main(EmbeddedRest.java:49) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> >> sun.reflect.DelegatingMethodAccessorImp
[GitHub] activemq-artemis pull request: Large message database
Github user mtaylor commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/502#discussion_r62008135 --- Diff: artemis-jdbc-store/src/main/java/org/apache/activemq/artemis/jdbc/store/file/JDBCSequentialFileFactory.java --- @@ -0,0 +1,255 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.activemq.artemis.jdbc.store.file; + +import java.io.File; +import java.lang.ref.WeakReference; +import java.nio.ByteBuffer; +import java.sql.Connection; +import java.sql.Driver; +import java.sql.PreparedStatement; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import java.util.concurrent.Executor; + +import org.apache.activemq.artemis.core.io.SequentialFile; +import org.apache.activemq.artemis.core.io.SequentialFileFactory; +import org.apache.activemq.artemis.core.server.ActiveMQComponent; +import org.apache.activemq.artemis.jdbc.store.JDBCUtils; +import org.apache.activemq.artemis.jdbc.store.file.sql.SQLProvider; +import org.apache.activemq.artemis.journal.ActiveMQJournalLogger; + +public class JDBCSequentialFileFactory implements SequentialFileFactory, ActiveMQComponent { + + private Connection connection; + + private String connectionUrl; + + private final Driver driver; + + private boolean started; + + private final String tableName; + + private List files; + + private PreparedStatement selectFileNamesByExtension; + + private Executor executor; + + private SQLProvider sqlProvider; + + private Map fileLocks = new HashMap<>(); + + public JDBCSequentialFileFactory(final String connectionUrl, +final String tableName, +final String className, +Executor executor) throws Exception { + this.connectionUrl = connectionUrl; + this.executor = executor; + this.tableName = tableName.toUpperCase(); + + files = new ArrayList<>(); + sqlProvider = JDBCUtils.getSQLProvider(JDBCUtils.getDriver(className).getClass().getCanonicalName(), tableName); + driver = JDBCUtils.getDriver(className); + } + + public Connection getConnection() { + return connection; + } + + @Override + public SequentialFile createSequentialFile(String fileName) { + try { + fileLocks.putIfAbsent(fileName, new Object()); + JDBCSequentialFile file = new JDBCSequentialFile(this, fileName, sqlProvider, executor, fileLocks.get(fileName)); + files.add(file); + return file; + } + catch (Exception e) { + ActiveMQJournalLogger.LOGGER.error("Could not create file", e); + } + return null; + } + + @Override + public int getMaxIO() { + return 1; + } + + @Override + public List listFiles(String extension) throws Exception { + List fileNames = new ArrayList<>(); + + selectFileNamesByExtension.setString(1, extension); + try (ResultSet rs = selectFileNamesByExtension.executeQuery()) { + while (rs.next()) { +fileNames.add(rs.getString(1)); + } + } + return fileNames; + } + + @Override + public boolean isSupportsCallbacks() { + return true; + } + + @Override + public void onIOError(Exception exception, String message, SequentialFile file) { + + } + + // Methods lifted from NIO Impl + @Override + public ByteBuffer allocateDirectBuffer(final int size) { + // Using direct buffer, as described on https://jira.jboss.org/browse/HORNETQ-467 --- End diff --
[GitHub] activemq-artemis pull request: Large message database
Github user mtaylor commented on a diff in the pull request: https://github.com/apache/activemq-artemis/pull/502#discussion_r62007801 --- Diff: tests/integration-tests/src/test/java/org/apache/activemq/artemis/tests/integration/xa/BasicXaRecoveryTest.java --- @@ -101,7 +101,7 @@ public void setUp() throws Exception { addressSettings.clear(); if (storeType == StoreConfiguration.StoreType.DATABASE) { - configuration = createDefaultJDBCConfig().setJMXManagementEnabled(true); --- End diff -- Not sure what you mean. I added boolean to createDefaultJDBCConfig to allow the test to pass in whether or not t configure the server to set up a netty connector (previously it always used Netty). It should be using JDBC for the test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---