mcvsubbu commented on a change in pull request #5221: Add a new server api for
download of segments.
URL: https://github.com/apache/incubator-pinot/pull/5221#discussion_r405190350
##########
File path:
pinot-server/src/main/java/org/apache/pinot/server/api/resources/TablesResource.java
##########
@@ -175,4 +183,41 @@ public String getCrcMetadataForTable(
}
}
}
+
+ @GET
+ @Produces(MediaType.APPLICATION_OCTET_STREAM)
+ @Path("/tables/{tableName}/segments/{segmentName}")
+ @ApiOperation(value = "Download a segment", notes = "Download a segment in
zipped tar format")
+ public Response downloadSegment(
+ @ApiParam(value = "Name of the table with type REALTIME OR OFFLINE",
required = true, example = "myTable_OFFLINE") @PathParam("tableName") String
tableName,
+ @ApiParam(value = "Name of the segment", required = true)
@PathParam("segmentName") @Encoded String segmentName,
+ @Context HttpHeaders httpHeaders)
+ throws Exception {
+ LOGGER.info("Get a request to download segment {} for table {}",
segmentName, tableName);
+ TableDataManager tableDataManager = checkGetTableDataManager(tableName);
+ SegmentDataManager segmentDataManager =
tableDataManager.acquireSegment(segmentName);
+ if (segmentDataManager == null) {
+ throw new WebApplicationException(String.format("Table %s segments %s
does not exist", tableName, segmentName),
+ Response.Status.NOT_FOUND);
+ }
+ try {
+ String tableDir = tableDataManager.getTableDataDir();
+ String tarFilePath =
TarGzCompressionUtils.createTarGzOfDirectory(tableDir + "/" + segmentName);
Review comment:
We may need to limit the number of simultaneous execution of this line. The
server is serving queries, and can get overwhelmed and affect the response time
if too many of these are execting.
I am fine with noting this as a TODO here for now, and re-visiting if it
becomes a problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
