swirl
Home Software Blog Wallpapers Webtools
Calling multiple REST APIs in Java
Monday 08, August 2022   |   Post link

Introduction

Yesterday a team-member asked me what I think is the best way to call two REST APIs in parallel with the additional requirement that the final output is computed based on the result obtained by both.

The optimum way to achive the above would be to make two parallel calls to the APIs, wait until both calls returns and then use the results to prepare the final output. For example if one of the API takes 1 second and the other takes 0.5 seconds, calling both will result in both results being available after 1 second. The next question is how do we do this?

There are multiple ways the above code can be written. This blog post discusses three approaches, one of which requires Java 11.

  • Using Apache HttpClient with an executor
  • Using Apache Async HttpClient
  • Using Java 11's HttpClient

Using Apache HttpClient with an executor

private static void requestWithApacheHttpClient(String url1, String url2) throws IOException {
    CountDownLatch latch = new CountDownLatch(2);
    AtomicInteger n1 = new AtomicInteger();
    AtomicInteger n2 = new AtomicInteger();

    m_executor.execute(()-> {
        try {
            HttpGet getReq = new HttpGet(url1);
            CloseableHttpResponse resp = m_closableHttpClient.execute(getReq);
            InputStream is = resp.getEntity().getContent();
            String result = EntityUtils.toString(resp.getEntity());
            n1.set(Integer.parseInt(result));
            latch.countDown();
        }
        catch(Exception e) {
            e.printStackTrace();
        }
    });

    m_executor.execute(()-> {
        try {
            HttpGet getReq = new HttpGet(url2);
            CloseableHttpResponse resp = m_closableHttpClient.execute(getReq);
            InputStream is = resp.getEntity().getContent();
            String result = EntityUtils.toString(resp.getEntity());
            n2.set(Integer.parseInt(result));
            latch.countDown();
        }
        catch(Exception e) {
            e.printStackTrace();
        }
    });

    try {
        latch.await();
        int n = n1.get() + n2.get();
        System.out.printf("\nCurrentMS: %d, result: %d", System.currentTimeMillis(), n);
    }
    catch(Exception e) {
        e.printStackTrace();
    }
}

Using Apache Async HttpClient

private static void requestWithApacheAsyncHttpClient(String url1, String url2) throws Exception {
    Future f1 = m_apacheAsyncHttpClient.execute(new HttpGet(url1), null);
    Future f2 = m_apacheAsyncHttpClient.execute(new HttpGet(url2), null);

    String r1 = EntityUtils.toString(f1.get().getEntity());
    int n1 = Integer.parseInt(r1);

    String r2 = EntityUtils.toString(f2.get().getEntity());
    int n2 = Integer.parseInt(r2);

    int n = n1 + n2;
    System.out.printf("\nCurrentMS: %d, result: %d", System.currentTimeMillis(), n);
}

Using Java 11's HttpClient

private static void requestWithJava11HttpClient(String url1, String url2) {
    HttpRequest req1 = HttpRequest.newBuilder()
            .uri(URI.create(url1))
            .timeout(Duration.ofMillis(5000))
            .GET()
            .build();

    HttpRequest req2 = HttpRequest.newBuilder()
            .uri(URI.create(url2))
            .timeout(Duration.ofMillis(5000))
            .GET()
            .build();

    CompletableFuture> f1;
    f1 = m_javaClient.sendAsync(req1, HttpResponse.BodyHandlers.ofString());

    CompletableFuture> f2;
    f2 = m_javaClient.sendAsync(req2, HttpResponse.BodyHandlers.ofString());

    f1.thenCombine(f2, (s1, s2) -> {
        int n1 = Integer.parseInt(s1.body());
        int n2 = Integer.parseInt(s2.body());
        return (n1 + n2);
    }).thenAccept((n) -> {
        try {
            m_totalResponses.incrementAndGet();
            System.out.printf("\nCurrentMS: %d, result: %d", System.currentTimeMillis(), n);
        }
        catch(Exception e) {
            e.printStackTrace();
        }
    });
}    

Testing method

Each method was called 500 times and followed by a gap of 5 seconds which was followed by another set of 500 calls, in total 2500 calls were made to both REST APIs. The REST API was written in Python and is as simple as:

@app.route('/n1', methods = ['GET'])
def getN1():    
    return "1";
    
@app.route('/n2', methods = ['GET'])
def getN2():    
    return "2";    

Testing results

Test Peak Threads Total Time (ms)
Apache HttpClient (100 threads) 111 35991
Apache HttpClient (1000 threads) 1010 33956
Apache Async Http Client 35793 20
Apache HttpClient (1000 threads) (JRE8) 1019 36679
Java 11 Http Client 413 22636
Comparison Chart

Conclusion

Java 11's native HttpClient is almost 50% better than using Apache's HttpClient as well as Async HttpClient. This in itself is a great reason to move to Java11 if you're still using Java 8. Code is available on GitHub.



Categories: Java (8)

Comments

Posts By Year

2022 (9)
2021 (5)
2020 (12)
2019 (6)
2018 (8)
2017 (11)
2016 (6)
2015 (17)
2014 (2)
2013 (4)
2012 (2)

Posts By Category

.NET (4)
.NET Core (2)
ASP.NET MVC (4)
AWS (4)
AWS API Gateway (1)
Android (1)
Apache Camel (1)
Architecture (1)
Azure (2)
Book review (3)
Business (1)
C# (3)
C++ (2)
CloudHSM (1)
Containers (4)
Corporate culture (1)
Database (3)
Database migration (1)
Desktop (1)
Docker (1)
DotNet (2)
DotNet Core (2)
Entity Framework (3)
Git (2)
IIS (1)
JDBC (1)
Java (8)
Kubernetes (1)
Lambda (1)
Learning (1)
Life (7)
Linux (1)
Lucene (1)
Multi-threading (1)
Music (1)
OData (1)
Office (1)
PHP (1)
Photography (1)
PowerShell (2)
Programming (26)
Rants (5)
SQL (2)
SQL Server (1)
Security (2)
Software Engineering (1)
Software development (1)
Solr (1)
Sql Server (2)
Storage (1)
T-SQL (1)
TDD (1)
TSQL (5)
Tablet (1)
Technology (1)
Test Driven (1)
Unit Testing (1)
Unit Tests (1)
Utilities (3)
VC++ (1)
VMWare (1)
VSCode (1)
Visual Studio (2)
Wallpapers (1)
Web API (2)
Win32 (1)
Windows (9)
XML (2)

Posts By Tags

.NET(6) API Gateway(1) ASP.NET(4) AWS(2) Adults(1) Advertising(1) Android(1) Anti-forgery(1) Asynch(1) Authentication(2) Azure(2) Backup(1) Beliefs(1) BlockingQueue(1) Book review(2) Books(1) Busy(1) C#(4) C++(3) CLR(1) CORS(1) CSRF(1) CTE(1) Callbacks(1) Camel(1) Certificates(1) Checkbox(1) CloudHSM(1) Cmdlet(1) Company culture(1) Complexity(1) Consumer(1) Consumerism(1) Containers(3) Core(2) Custom(2) DPI(1) Data-time(1) Database(4) Debugging(1) Delegates(1) Developer(2) Dockers(2) DotNetCore(3) EF 1.0(1) Earphones(1) Elastic Search(1) Encrypted(1) Entity framework(1) Events(1) File copy(1) File history(1) Font(1) Git(2) HierarchyID(1) IIS(1) Installing(1) Intelli J(1) JDBC(1) JSON(1) JUnit(1) JWT(1) Java(2) JavaScript(1) Kubernetes(1) Life(1) LinkedIn(1) Linux(1) Localization(1) Log4J(1) Log4J2(1) Lucene(1) MVC(4) Management(2) Migration history(1) Mobile Apps(1) Modern Life(1) Money(1) Music(1) NGINX(1) NTFS(1) NUnit(2) OData(1) OPENXML(1) Objects(1) Office(1) OpenCover(1) Organization(1) PHP(1) Paths(1) PowerShell(2) Producer(1) Programming(1) Python(2) Quality(1) REDIS(2) REST(1) Runtimes(1) S3-Select(1) SD card(1) SLF4J(1) SQL(2) SQL Code-first Migration(1) SSH(2) Sattelite assemblies(1) School(1) Secrets Manager(1) Self reliance(1) Shell(1) Solr(1) Sony VAIO(1) Spirituality(1) Sql Express(1) System Image(1) TDD(1) TSQL(3) Table variables(1) Tables(1) Tablet(1) Ubuntu(1) Url rewrite(1) VMWare(1) VSCode(1) Validation(2) VeraCode(1) Wallpaper(1) Wallpapers(1) Web Development(4) Windows(2) Windows 10(2) Windows 2016(2) Windows 8.1(1) Work culture(1) XML(1) Yii(1) renew(1)