i'm fairly new to parquet format. my team uses this to submit data to be loaded to our enterprise data warehouse/data lake. i have two questions.
can i generally concatenate many parquet formatted files together to make one larger file? i get millions of small xml data files from mobile devices and want to convert each to parquet via an aws lambda to an s3 bucket. then i can sweep on a cadence and concatenate the files and submit to be loaded to the data lake. they don't like millions of submissions per day, or i would submit each individual file. secondly, i have several nginx access logs that i want to convert to parquet for loading to the same data lake. are there tools for easily converting these logs to parquet format?
