Hi,
I am writing a singleton class (Object in scala) which uses apache
httpclient(4.5.2) to send some file or some json content to a server and
return status to caller. Here is my class -
object HttpUtils{
protected val retryHandler = new HttpRequestRetryHandler() {
def retryRequest(exception: IOException, executionCount: Int,
context: HttpContext): Boolean = {
//retry logic
true
}
}
private val connectionManager = new
PoolingHttpClientConnectionManager()
// Reusing same client for each request that might be coming from
different threads .
// Is it correct ????
val httpClient = HttpClients.custom()
.setConnectionManager(connectionManager)
.setRetryHandler(retryHandler)
.build()
def restApiCall (url : String, rDD: RDD[SomeMessage]) : Boolean = {
// Creating new context for each request
val httpContext: HttpClientContext = HttpClientContext.create
val post = new HttpPost(url)
// convert RDD to text file using rDD.collect
// add this file as MultipartEntity to post
var response = None: Option[CloseableHttpResponse] // Is it correct
way of using it ???
try {
response = Some(httpClient.execute(post, httpContext))
val responseCode = response.get.getStatusLine.getStatusCode
EntityUtils.consume(response.get.getEntity) // Is it require ???
if (responseCode == 200) true
else false
}
finally {
if (response.isDefined) response.get.close
post.releaseConnection() // Is it require ???
}
}
def onShutDown = {
connectionManager.close()
httpClient.close()
}
}
Multiple threads (More specifically from spark streaming context) are
calling `restApiCall` method. I am relatively new to `scala` and `apache
httpClient`. I have to make frequent connections to only few fixed server
(i.e. 5-6 fixed URL's with different request parameters). I have also
written comments in above code as question. Please address them as well.
I went through multiple online resource but still not confident about it.
- Is it the best way to use http client in multi-threaded environment?
- Is it possible to keep live connections and use it for various requests
? Will it be beneficial in this case ?
- Am i using/releasing all resources efficiently ? If not please suggest.
- Is it good to use it in Scala or there exist some better library ?
Thanks for your help in advance.
--
Nishant Kumar
Bangalore, India
Mob: +91 80088 42030
Email: [email protected]
