> On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote: > > agents-audit/README.txt > > Lines 28 (patched) > > <https://reviews.apache.org/r/63552/diff/3/?file=1880895#file1880895line28> > > > > I assume, which this option, memory buffer won't be used.
Bosco, memory buffer is not removed, AuditFileCached provider is going to fetch data from the file in buffer sizes and send it AuditQueue only in the current pipeline of data flow. I shall details this in JIRA and we can review this as this is significant in what is support and what not. > On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote: > > agents-audit/src/main/java/org/apache/ranger/audit/utils/AuditWriter.java > > Lines 97 (patched) > > <https://reviews.apache.org/r/63552/diff/3/?file=1880900#file1880900line97> > > > > This class seems more HDFS specific. Should we just call it > > HDFSAuditWriter? > > > > I think, we have two variables: > > 1. FileSystem (Local File, HDFS, S3, etc) > > 2. Format/encoding (JSON, ORC, Parquet, Avro, etc.) > > > > We should be able to mix and match where possible. FileSystem API of Hadoop core is handling all the fileSystem, so it just the param value what is configured in the AuditFramework will determine the destination like HDFS,LocalFile, S3,etc) and Ranger AuditWriter will handle fileformat of the audit. > On Nov. 4, 2017, 9:50 p.m., Don Bosco Durai wrote: > > agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java > > Lines 154 (patched) > > <https://reviews.apache.org/r/63552/diff/3/?file=1880901#file1880901line154> > > > > Should we log and ignore this Exception This is done for purpose, so that the calling source can do the deffered count and log it. - Ramesh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63552/#review190104 ----------------------------------------------------------- On Nov. 17, 2017, 7:38 p.m., Ramesh Mani wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63552/ > ----------------------------------------------------------- > > (Updated Nov. 17, 2017, 7:38 p.m.) > > > Review request for ranger, Don Bosco Durai and Madhan Neethiraj. > > > Repository: ranger > > > Description > ------- > > RANGER-1837:Enhance Ranger Audit to HDFS to support ORC file format > > > Diffs > ----- > > agents-audit/README.txt PRE-CREATION > agents-audit/pom.xml c8bd1d8 > > agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java > 66d8504 > > agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditFileCacheProvider.java > 314b130 > > agents-audit/src/main/java/org/apache/ranger/audit/provider/AuditWriterFactory.java > PRE-CREATION > agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java > eff3824 > > agents-audit/src/main/java/org/apache/ranger/audit/queue/AuditFileCacheProviderSpool.java > 41513ba > > agents-audit/src/main/java/org/apache/ranger/audit/utils/AbstractAuditWriter.java > PRE-CREATION > agents-audit/src/main/java/org/apache/ranger/audit/utils/JSONWriter.java > PRE-CREATION > agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCFileUtil.java > PRE-CREATION > agents-audit/src/main/java/org/apache/ranger/audit/utils/ORCWriter.java > PRE-CREATION > agents-audit/src/main/java/org/apache/ranger/audit/utils/Writer.java > PRE-CREATION > pom.xml 589cd6a > src/main/assembly/hbase-agent.xml 3ebc334 > src/main/assembly/hdfs-agent.xml 5279a9a > src/main/assembly/hive-agent.xml ca65c80 > src/main/assembly/knox-agent.xml 8357d49 > src/main/assembly/plugin-atlas.xml fd98811 > src/main/assembly/plugin-kafka.xml 95855d9 > src/main/assembly/plugin-kms.xml 6d15f2a > src/main/assembly/plugin-solr.xml de30bfb > src/main/assembly/plugin-sqoop.xml d2bd69a > src/main/assembly/plugin-yarn.xml c6a48e8 > src/main/assembly/storm-agent.xml 64224ec > > > Diff: https://reviews.apache.org/r/63552/diff/4/ > > > Testing > ------- > > Testing done in local > > > Thanks, > > Ramesh Mani > >
