A Dynamic Management System of Synchronous-Asynchronous API Services
DOI:
https://doi.org/10.22399/ijcesen.4953Keywords:
API aggregation, synchronous-asynchronous hybrid, Redis, dynamic data ingestion, schema-less querying, KafkaAbstract
Modern applications increasingly rely on data from heterogeneous APIs, databases, and IoT sources, yet existing integration solutions struggle with latency, schema heterogeneity, data consistency, and fault tolerance. This paper proposes a hybrid synchronous-asynchronous API management system that enables low-latency interactive access and high-throughput background processing. The core innovation is automatic, schema-agnostic ingestion of JSON responses into endpoint-specific Redis tables, augmented with reception timestamps, enabling efficient time-series and historical querying without manual ETL or schema inference. The system exposes a REST/WebSocket interface for synchronous requests, a Kafka-based asynchronous pipeline for complex aggregations, and a SQL-like query layer over the stored data. Evaluation on a Kubernetes cluster with simulated heterogeneous APIs demonstrates median response times of 78 ms for synchronous aggregations (35–62% faster than GraphQL federation and REST aggregation baselines) and 99.2% data completeness under 30% concurrent source failure in asynchronous mode. Case studies in smart-city dashboards and healthcare monitoring illustrate reduced development effort and improved performance. The architecture bridges API consumption and database functionality, offering a lightweight alternative for real-time federated analysis and IoT integration.
References
[1] M. A. Baazizi, H. B. Lahmar, D. Colazzo, G. Ghelli, and C. Sartiani, "Schema inference for massive JSON datasets," in Extending Database Technology (EDBT), 2017, pp. 222-233.
[2] K. Banker, MongoDB in Action. Manning Publications, 2016.
[3] L. Byron, “GraphQL: A data query language,” Facebook Engineering Blog, Sep. 2016. [Online]. Available: https://engineering.fb.com/2016/09/12/data/graphql-a-data-query-language/
[4] R. Cappuzzo, P. Papotti, and S. Thirumuruganathan, "Creating embeddings of heterogeneous relational datasets for data integration tasks," in Proc. ACM SIGMOD Int. Conf. Management of Data, 2020, pp. 1335-1349.
[5] T. Dunning and E. Friedman, Streaming Architecture: New Designs Using Apache Kafka and MapR Streams. O'Reilly Media, 2016.
[6] GraphQL Foundation, “GraphQL Specification,” June 2018. [Online]. Available: https://graphql.org/
[7] R. T. Fielding, "Architectural styles and the design of network-based software architectures," Ph.D. dissertation, University of California, Irvine, 2000.
[8] M. Fowler, "BFF: Backend for Frontend," martinfowler.com, 2015. [Online]. Available: https://martinfowler.com/articles/backend-for-frontend.html
[9] E. Gallinucci, M. Golfarelli, and S. Rizzi, "Schema profiling of document-oriented databases," Information Systems, vol. 75, pp. 13-25, 2018.
[10] E. Huang, L. Xu, and T. Zhang, "TiDB: A Raft-based HTAP database," Proc. VLDB Endowment, vol. 13, no. 12, pp. 3072-3084, 2020.
[11] S. Idreos, K. Deng, T. Kraska, S. Madden, M. Stonebraker, and J. Yang, "The Data Civilizer system," in CIDR, 2019.
[12] K. Indrasiri and P. Siriwardena, Microservices for the Enterprise: Designing, Developing, and Deploying. Apress, 2021.
[13] J. Kreps, "Questioning the Lambda Architecture," O'Reilly Radar, 2014. [Online]. Available: https://www.oreilly.com/radar/questioning-the-lambda-architecture/
[14] J. Kreps, N. Narkhede, and J. Rao, "Kafka: A distributed messaging system for log processing," in Proc. NetDB, 2011, pp. 1-7.
[15] N. Marz and J. Warren, Big Data: Principles and Best Practices of Scalable Realtime Data Systems. Manning Publications, 2015.
[16] B. M. Michelson, "Event-driven architecture overview," Patricia Seybold Group, vol. 2, no. 12, pp. 10-1571, 2006.
[17] S. Mudgal, H. Li, T. Rekatsinas, A. Doan, Y. Park, G. Krishnan, and F. Naumann, "Deep learning for entity matching: A design space exploration," in Proc. ACM SIGMOD Int. Conf. Management of Data, 2018, pp. 19-34.
[18] S. Newman, Building Microservices: Designing Fine-Grained Systems. O'Reilly Media, 2015.
[19] S. Newman, Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith. O'Reilly Media, 2019.
[20] F. Özcan, Y. Tian, and P. Tözün, "Hybrid transactional/analytical processing: A survey," in Proc. ACM SIGMOD Int. Conf. Management of Data, 2017, pp. 1771-1775.
[21] A. Pavlo, G. Angulo, J. Arulraj, H. Lin, J. Lin, L. Ma, and M. Stonebraker, "Self-driving database management systems," in CIDR, 2017.
[22] E. Rahm and P. A. Bernstein, "A survey of approaches to automatic schema matching," The VLDB Journal, vol. 10, no. 4, pp. 334-350, 2001.
[23] C. Richardson, Microservices Patterns: With Examples in Java. Manning Publications, 2019.
[24] N. Shamgunov, "The MemSQL database: Technology and performance," in Proc. ACM SIGMOD Int. Conf. Management of Data, 2017, pp. 1737-1738.
[25] J. Strachan, "JSONata: JSON query and transformation language," Technical Report, 2017. [Online]. Available: https://jsonata.org/
[26] A. Vázquez-Ingelmo, J. Cruz-Benito, and F. J. García-Peñalvo, "Improving the OEEU's data-driven technological ecosystem's interoperability with GraphQL," in Proc. 7th Int. Conf. Technological Ecosystems for Enhancing Multiculturality, 2020, pp. 1-8.
[27] E. Wittern, A. Cha, J. C. Davis, G. Baudart, and L. Mandel, "An empirical study of GraphQL schemas," in Int. Conf. Service-Oriented Computing, 2019, pp. 3-19.
[28] G. Young, "CQRS Documents," Technical Report, 2010. [Online]. Available: https://cqrs.files.wordpress.com/2010/11/cqrs_documents.pdf
[29] A. Caraffa, "Synchronous and Asynchronous APIs," OpenAPI Blog, Dec. 2025. [Online]. Available: https://openapi.com/blog/synchronous-and-asynchronous-apis
[30] Gravitee.io, "Gravitee: API Management Platform for APIs, Events & Agents," 2025. [Online]. Available: https://www.gravitee.io/
[31] Nordic APIs, "Top 10 API Gateways in 2025," Nordic APIs, Jun. 2025. [Online]. Available: https://nordicapis.com/top-10-api-gateways-in-2025
[32] L.Cogan et al, Redis, "Redis 8.2 GA: Performance, Efficiency, and New Commands," Redis Blog, Aug. 2025. [Online]. Available: https://redis.io/blog/redis-82-ga
[33] L. Lakshika, et al. "Async Messaging at Scale: Kafka + Redis Streams + Spring Boot (20,000 RPS Blueprint — 2025 Edition)," Stackademic, Nov. 2025. [Online]. Available: https://blog.stackademic.com/async-messaging-at-scale-kafka-redis-streams-spring-boot-20-000-rps-blueprint-2025-7d19a3f1f746
[34] C. Reddy Kasaram, et al. "Harnessing Asynchronous Patterns with Event Driven Kafka and Microservices Architectures," ResearchGate, Oct. 2023 (published version 2025). [Online]. Available: https://www.researchgate.net/publication/397530105
[35] A. Jain, et al."Building a High-Frequency Trading System With Hybrid Strategy (Redis & InfluxDB): From 10ms to Sub-Millisecond Latency," Medium, Nov. 2025. [Online]. Available: https://vardhmanandroid2015.medium.com/building-a-high-frequency-trading-system-with-hybrid-strategy-redis-influxdb-from-10ms-to-85716febefcb
[36] J. Boyer, et al. "Reference Architecture - Event-Driven Solutions in Hybrid Cloud," GitHub Pages, 2025 (ongoing). [Online]. Available: https://jbcodeforce.github.io/eda-studies/concepts/eda
[37] Koukaras, Paraskevas et al. “Data Integration and Storage Strategies in Heterogeneous Analytical Systems: Architectures, Methods, and Interoperability Challenges.” Inf. 16 (2025): 932.
[38] Sserujongi, Richard et al. “Design and Evaluation of a Scalable Data Pipeline for AI-Driven Air Quality Monitoring in Low-Resource Settings.” ArXiv abs/2508.14451 (2025): n. pag.
[39] Arafat, Jahidul et al. “Next-Generation Event-Driven Architectures: Performance, Scalability, and Intelligent Orchestration Across Messaging Frameworks.” ArXiv abs/2510.04404 (2025): n. pag.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
