CVE-2022-45910: Apache ManifoldCF: LDAP Injection Vulnerability - ActiveDirectory Authorities

2022-12-06 Thread Markus Schuch
Description: Improper neutralization of special elements used in an LDAP query ('LDAP Injection') vulnerability in ActiveDirectory and Sharepoint ActiveDirectory authority connectors of Apache ManifoldCF allows an attacker to manipulate the LDAP search queries (DoS, additional queries, filter

Re: Manifoldcf server Error

2019-12-20 Thread Markus Schuch
oducing the error with Public sites of the same > domain and on a different server(DEV), it was successful, with no > error.Also there was no any postgres related error. > > Can it depends observer related configurations like Firewall etc, as > this case include some fi

Re: Manifoldcf server Error

2019-12-20 Thread Markus Schuch
ith 8 cores CPU. > > Can you suggest some solution. > > Thanks > Priya > > On Fri, Dec 20, 2019 at 2:23 PM Markus Schuch <mailto:markus_sch...@web.de>> wrote: > > Hi Priya, > > your manifoldcf JVM suffers from high garbage collection pressure: &

Re: Manifoldcf server Error

2019-12-20 Thread Markus Schuch
Hi Priya, your manifoldcf JVM suffers from high garbage collection pressure: java.lang.OutOfMemoryError: GC overhead limit exceeded What is your current heap size? Without knowing that, i suggest to increase the heap size. (java -Xmx...) Cheers, Markus Am 20.12.2019 um 09:02 schrieb Priya

Re: Manifold with OpenJDK

2019-10-16 Thread Markus Schuch
Hi Praveen, we use openjdk 8 in dockered red hat linux for 2 years now and didn't have problems with it. We had one minor issue when we migrated: the image processing capabilities of openjdk are somehow different from Oracle JDK. One of our connectors creates image thumbnails and on openjdk

Re: Solr Connector rejects Webpages when using TikaExtractor and SolrJ mode

2019-08-23 Thread Markus Schuch
type, because that may be of interest at indexing > time.  So a filter specified in the solr connector should always be > against the original mime type and not the modified one. > > Let me make that change. > > Karl > > > On Fri, Aug 23, 2019 at 6:31 A

Re: Solr Connector rejects Webpages when using TikaExtractor and SolrJ mode

2019-08-23 Thread Markus Schuch
x and it should just default to > allowing "text/plain" with no other options accepted. > > Karl > > > On Fri, Aug 23, 2019 at 2:17 AM Markus Schuch <mailto:markus_sch...@web.de>> wrote: > > Hi Karl, > > what do i have to do to make tika decl

Re: Solr Connector rejects Webpages when using TikaExtractor and SolrJ mode

2019-08-23 Thread Markus Schuch
Solr connector. > > Karl > > > On Thu, Aug 22, 2019 at 5:44 PM Markus Schuch <mailto:markus_sch...@web.de>> wrote: > > Hi, > > i am playing around with the solrj mode of the solr output connector, to > avoid running tika extrac

AW: Error: Unexpected jobqueue status - record id X, expecting active status, saw 4 (MySQL compatible Database)

2019-06-06 Thread Markus Schuch
mail ? Did you change something on your MCF configuration to fix this ? Thanks, Best regards, Olivier Le 13 févr. 2019 à 13:58, Markus Schuch mailto:markus.sch...@deutschebahn.com>> a écrit : Hi Karl, we set the diagnostigs logger to level debug. I will get back when the error

Re: Where and how is ManifoldCF used in production?

2019-03-21 Thread Markus Schuch
Hi, we use MCF as part of a homegrown Solr based enterprise search solution (1,1M docs). We use some of the shipped connectors (sharepoint, livelink, web) but also implemented a lot of ourselves for connecting proprietary repositories. We deploy on AWS on EC2 and RDS Aurora MySQL. Cheers,

AW: Error: Unexpected jobqueue status - record id X, expecting active status, saw 4 (MySQL compatible Database)

2019-02-13 Thread Markus Schuch
Hi Karl, we set the diagnostigs logger to level debug. I will get back when the error occurs again. Cheers, Markus Pflichtangaben anzeigen Nähere Informationen zur Datenverarbeitung im DB-Konzern finden Sie

Error: Unexpected jobqueue status - record id X, expecting active status, saw 4 (MySQL compatible Database)

2019-02-12 Thread Markus Schuch
se/CONNECTORS-1180 [2] https://issues.apache.org/jira/browse/CONNECTORS-590 [3] https://issues.apache.org/jira/browse/CONNECTORS-246 -- Markus Schuch Web Business (T.IPB 26) DB Systel GmbH Jürgen-Ponto-Platz 1, 60329 Frankfurt a. Main Pflichtangaben

Re: Using File System Repository Connector for a Sample Crawl in Windows Environment

2018-05-01 Thread Markus Schuch
Hi Irindu, i suppose you left the text field in the "root path" column (next to the "Add" Button) empty, so the root path is the execution directory of your ManifoldCF instance. Instead of using the match rules fields you need to enter your desired crawl root directory as "root path". The

Re: Amazon RDS for PostgreSQL Support

2017-12-16 Thread Markus Schuch
that multiple records get updated in one query; this is > usually helpful but when each update is expensive you could wind > up with locking causing delays. > > __ __ > > Karl > > __ __ > > __ __ > > On Thu, Nov 30, 2017 at 2:33 AM

AW: Amazon RDS for PostgreSQL Support

2017-12-15 Thread Markus Schuch
causing delays. Karl On Thu, Nov 30, 2017 at 2:33 AM, Markus Schuch <markus.sch...@deutschebahn.com<mailto:markus.sch...@deutschebahn.com>> wrote: Hi Karl, we disabled autovacuum The stats table show there was no autovac since then. The long running queries still occur. There are n

AW: Amazon RDS for PostgreSQL Support

2017-11-29 Thread Markus Schuch
Hi, since nobody responded, we started to experiment. Setup: AWS RDS Postgres 9.6.3 db.m4.xlarge (4 cores, 16 GB RAM) provisioned iops 4000 Indexing Performance for most jobs is pretty good, except a for a bigger sharepoint crawl (~200.000 docs). We are seeing a lot of long running queries

Amazon RDS for PostgreSQL Support

2017-10-21 Thread Markus Schuch
Hi everyone, does anybody use ManifoldCF on AWS RDS for PostgreSQL? We think about trying RDS and we would be very happy if some folks could share their experiences. e.g. used instance types, count of documents, known issues, limitations, performance Many thanks in advance, Markus

AW: Sharepoint Repository Connector: Metadata Changes not causing re-index library or list items

2017-10-18 Thread Markus Schuch
Hi Karl, after some time i am finally looking into this again. The lookup field refers to an attribute of another list. Therefore changes to that attribute do not affect the modified date of the document or the document collection. Do you see any possible solution in the connector for that

Re: SharePoint crawler ArrayIndexOutOfBoundException in log

2017-03-17 Thread Markus Schuch
Hi, i think this may caused by https://issues.apache.org/jira/browse/HTTPCLIENT-1715 which was fixed in httpclient 4.5.2 There is a very similar stacktrace in https://issues.apache.org/jira/browse/HTTPCLIENT-1686 which is also linked to HTTPCLIENT-1715. Cheers, Markus Am 17.03.2017 um

Re: [Webcrawler Connector] Feature for ignoring meta/rel robots tags/attributes

2017-02-27 Thread Markus Schuch
; metadata/rel", and that the implementation be backwards compatible. > > Thanks, > Karl > > > On Sat, Feb 25, 2017 at 5:02 PM, Markus Schuch <markus_sch...@web.de > <mailto:markus_sch...@web.de>> wrote: > > Hi, > > what do you think about adding the possibility

Re: webcrawler connector and dns lookups behind corporate http proxy

2016-10-11 Thread Markus Schuch
On Mon, Oct 10, 2016 at 3:44 PM, Markus Schuch <markus_sch...@web.de[mailto:markus_sch...@web.de]> wrote:Hi @ the lovely mcf community out there,   in our setup we run manifoldcf (2.3) behind a corporate http proxy server and we try to crawl specific web pages in the internet.

webcrawler connector and dns lookups behind corporate http proxy

2016-10-10 Thread Markus Schuch
Hi @ the lovely mcf community out there,   in our setup we run manifoldcf (2.3) behind a corporate http proxy server and we try to crawl specific web pages in the internet.   We run into java.net.UnknownHostException because the connector tries to resolve the ip of the hostname. This fails,

Re: E-Mail notifications over SMTP

2016-04-15 Thread Markus Schuch
;_attribute_name":"port"}],"properties":[{"_value_":"","_attribute_name":"protocol","_attribute_value":"smtp"},{"_value_":"","_attribute_name":"type","_attribute_value"

Re: E-Mail notifications over SMTP

2016-04-13 Thread Markus Schuch
ample. > >protocol=smtp; type=transport; class=com.sun.mail.smtp.SMTPTransport; >vendor=Sun Microsystems, Inc.; > > >Anyhow, I'm more than willing to update the connector but it would be >great >to know what I should actually do to it first. > >Karl > > >

E-Mail notifications over SMTP

2016-04-13 Thread Markus Schuch
Hi, i wanted to try the email notification connector to send emails on job aborts. But couldn't get it working with our mx server. The email notification connector only offers POP3 or IMAP protocol. In my world these are for receiving emails. For sending i need SMTP. How am I supposed to

Aw: Re: Should a document with an empty version string always be reingested?

2016-03-04 Thread Markus Schuch
quot;.  Instead, your connector must signal its disposition of any document using the IProcessActivity methods available for that purpose.   Can you describe in more detail what you are doing here? (a) Is this a custom connector? (b) Was it developed on 1.7 or before? (c) Are you trying to run it on 1.10 or o

Should a document with an empty version string always be reingested?

2016-03-04 Thread Markus Schuch
Hi,   we ran on MCF 1.7 for quite a while and in this environment a document send to the ingestion pipeline together with an empty version string was always reingested. On MCF 2.3 this is no longer the case.   I found https://issues.apache.org/jira/browse/CONNECTORS-1153 and may be the 1.7

Some Questions about Migration from ManifoldCF 1.x to 2.x

2016-02-24 Thread Markus Schuch
Hi, we plan to migrate from ManifoldCF 1.7.1 to ManifoldCF 2.3. Is there a migration guide available? We want to run ManifoldCF 2.3 on Java 8 in a Servlet-API 3.0 container (Tomcat 7), which is - for now - predefined by our OPs. Is this possible? Or is Servlet now API 3.1 required? (we notices

OutOfMemoryError on email notifications

2015-12-13 Thread Markus Schuch
Hi, i just tried the relatively new email notification feature with version 2.2 and keep running into an OOME when notifications are about to be sent: agents process ran out of memory - shutting down java.lang.OutOfMemoryError: Java heap space     at java.util.Arrays.copyOf(Unknown Source)  

Deadlock in JDBC ConnectionPool

2015-06-23 Thread Markus Schuch
Hi, i tested CONNECTORS-1202 (stale connection check) and i encounter a dead lock in the connection pool when i simulate a database outage. I use ManifoldCF 1.7.1 with CONNECOTRS-1202 patch applied an MySQL as Database. Steps for reproduction : 1. Start ManifoldCF with a limited

Renaming Connector Classes

2015-05-20 Thread Markus Schuch
Hi, i need to rename some connector classes. When starting MCF with the renamed classes, i obviously stumbles over the old connections using the old classnames. It would be nice to be able to migrate the old connections to the new classnames. I found the commandsSynchronizeConnectors,

Health Check of database connections

2015-05-20 Thread Markus Schuch
Hi,   it seems that manifoldcf can not re-establish the database connection, when the database connection was lost. For example on MySQL manifoldcf stops working when we shutdown and start the database the database. Database exception: SQLException doing query (08): Could not send query: Last

Re: Renaming Connector Classes

2015-05-20 Thread Markus Schuch
to do the following: (1) BEFORE renaming the class, delete all jobs and connections that refer to that connector. (2) UNREGISTER the connector class (there's a command-line utility for that) (3) BUILD your new class (4) REBUILD everything you deleted Karl On Wed, May 20, 2015 at 8:56 AM, Markus

Re: Document components

2014-11-25 Thread Markus Schuch
Hi Karl,   the patch for CONNECTORS- fixes the cleanup issue.   Another question about primary documents and their components:   I have ingested a primary document with some components. During the next processing the primary document should no longer be indexed, but the sub components of it

Document components

2014-11-21 Thread Markus Schuch
Hi, is there any example implementation of the new document component feature invented with CONNECTORS-989? I read the section Document components in [0] but i still do not know how to actually write a repository connector that ingests multiple documents originating from a single document of a

Re: Document components

2014-11-21 Thread Markus Schuch
Hi Karl,   as i already documented in CONNECTORS-1110, the provided patch solved the problem.   Another question about an implementation detail when using components: Is it admissible to mix calls of ingestDocumentWithException(...) with and without the component identifier argument? E.g. call

Re: MariaDB Support

2014-10-24 Thread Markus Schuch
Hi, i opened CONNECTORS-1083 [0] with a patch for the new MariaDB database implementation. Regards, Markus -- [0] https://issues.apache.org/jira/browse/CONNECTORS-1083

MariaDB Support

2014-10-23 Thread Markus Schuch
Hi, anybody tried to run manifoldcf on top of MariaDB? (MysSQL Fork with LGPL license) Since MariaDB is documented as binary compatible i think changing the jdbc driver class should be the only change. Is it possible to change the mysql driver class name with a system property? Regards,

Re: Migration from mcf 1.1 to 1.7

2014-10-23 Thread Markus Schuch
Hi Karl, we encountered an issue when migrating on a derby database. (test environment running on derby) Is derby migration supported? (may be not in scope, since derby should be used for production?) org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database exception: SQLException

Migration from mcf 1.1 to 1.7

2014-10-20 Thread Markus Schuch
Hi users, we plan to switch from mcf 1.1 to 1.7. Are there any resources that sum up an describe the schema changes and backward compatibility for existing configurations? We have the the following questions: How are the schema changes handled? Is it necessary to start on an empty

Does ManifoldCF support distributed search with Apache Solr?

2013-11-13 Thread Markus Schuch
Hi, we are using Manifold CF with Apache Solr 4.1. We installed the ManifoldCFSearchComponent. Now we would like to split our Index into Shards. In the Code of the ManifoldCFSearchComponent there is the statement if (!params.getBool(COMPONENT_NAME, true) || params.get(ShardParams.SHARDS) !=

Re: How to map the atlassian confluence security model to manifoldcf

2013-05-30 Thread Markus Schuch
, I strongly suspect that real Atlassian sysadmins do things more like Example 2 than Example 1. What do you think? Karl On Sat, May 25, 2013 at 8:20 PM, Markus Schuch markus_sch...@web.de mailto:markus_sch...@web.de wrote: Hi Karl, no need

Re: How to map the atlassian confluence security model to manifoldcf

2013-05-25 Thread Markus Schuch
, May 24, 2013 at 6:57 PM, Markus Schuch markus_sch...@web.de wrote:Hi, we are currently writing a repository connector for confluence. We are using the solr output connection on Solr 4.x. Seeding, versioning, processing works already and now we have to face security. Compared to the already