Project Topics
Get Complete Project Material File(s) Now! »
The Evolution of Interface Description Formats
Introduction
« The Internet is slowly changing almost every aspect of our society, and a major part of this evolution is rooted in how we develop software, something that has been slowly evolving for over the last 15 years, in the form of what is known as a Application Programming Interfaces, or simply called API [96]. » – Kin Lane, The API Evangelist.
Man invented languages to convey the knowledge, and devised contracts to formalize business interactions. The concept of contracts is also found in electronic interactions. Standards from EDI (Electronic Data Interchange) [72] to XML [26], and from WSDL to Swagger [94] serve the purpose of specifying contracts among the interacting parties at various levels. In the context of web services, contracts are commonly known as service descriptions, which specify how a web service can be invoked by another service or system.
The primary purpose of a service description is to communicate service integration rules. A structured description makes it more convenient, especially for bad commu-nicators to clearly specify how a service can be called by another service or system. Probably the biggest advantage a structured description offers is the capability to be read by machines, which enables a service for automatic discovery, interactive docu-mentation, client/server side code generation, service mocking, testing and monitoring, search, reusability, interoperability and copyright [95], besides specifying a contract.
The Need of A Service Interface Description
In service oriented architecture, the role of a service interface description is to precisely indicate what a service does, i.e., the functions of a service, – where it resides, i.e., the web addresses of a service functions, and – how to invoke it, i.e., the information about the data formats and protocols to use the functions of a service.
The need of service descriptions vary from the nature of communication required to consume a service. This requirement may be classified for different environments as:
Nice-To-Have for Individuals and Small Teams: A developer creating web or mobile applications for individual use does not need an API description. Same is the situation while working within or with small teams because of the availability of easy and clear communication among the team members. However, having a service description is always good, and can help in service oriented activities such as testing and client code generation.
Should-Have for Enterprise: A typical enterprise system is a collection of sep-arated web applications which are heterogeneously developed, and are often re-quired to integrate with others enterprise systems, managed by several different companies. Defining service contracts can save valuable time and resources in such an environment.
Must-Have for External or third Party Integrations: Service descriptions have become absolutely essential in the recent times of XaaS (Everything as a Service). Contacting to people developing services is not always possible, while sometimes it is required to integrate a legacy application which runs, but has not been maintained by anyone, and it becomes a pain to find out a way to communicate with it.
Description Versus Documentation
A typical web service comes with two types of descriptions. The first one is structured (mostly machine readable) description like WSDL, and the other one is a description in natural language, also known as the documentation of a web service. The machine readable description is a formal document that contains the building blocks of a ser-vice, while the documentation acts as a guide for developers for using those building
blocks. For example, the Amazon E-Commerce Web Service (referred to subsequently as AECS) comes with a WSDL document [163], that defines AECS operations and data structures, as well as additional documents (e.g., [10]) to guide the developers about accessing the AECS. This documentation informally specifies the precise sequence in which the operations of AECS must be invoked to perform desired business processes. Without this sequence, the WSDL description is just like a library of independent operations, which are synchronous atomic (request-response) service invocations.
The Evolution
With the recent advent of smart devices and home-automation solutions, it has become much more common for software applications to directly communicate with each other. Consequently, web services, the key enabler of the internet based communications for a long time, have also been evolved to support the emerging service architectures, such as mobile internet. In this section, we will look at the evolution of web service interfaces from two aspects: description languages and emerging formats as shown in Figure 3.1.
Description Languages
The majority of the interface description formats are based on one of the four languages: XML [26], JSON [46], YAML [19] and Markdown [76]. The first two are focused on the exchange of data among services, while the last two aim to improve the readability of an interface description. To show the difference among the encoding styles of all four languages, a simple Person object with three fields: Id (integer), Name (string) and Department (string) is shown in Figure 3.2.
XML
Since early 90s, HTML [74] has been instrumental with its structural markup for web documents to display human-readable information. Extensible Markup Language (XML) [26] took the document markup to another level by introducing machine-readable semantic markup that could specify the meaning of an individual element at its own [92]. XML is simple, extensible, interoperable and a text-based format that evolved from Standard Generalized Markup Language (SGML) [73]. The first breed of web services used WSDL as its service description document and SOAP as the way to communicate over HTTP. Both WSDL and SOAP are based on XML, and have dom-inated the service oriented space for many years. The rise of mobile platforms and the inability of SOAP (with large XML data) to cope with mobile challenges [158], directed the SOA community to search for alternatives. Consequently, several new service de-scription formats emerged to fulfill the requirements of modern service architectures. Interestingly, none of the emerging formats use XML as its underlying language.
JSON
JSON (JavaScript Object Notation) is a data exchange language used largely in recent service oriented activities. An indication of the popularity of JSON is that most of the description formats introduced in the last five years, are powered by JSON. JSON is a text notation better suited to data-interchange while keeping the advantages of XML. The instant interpretation of JSON by JavaScript interpreters of the modern browsers eliminates any additional parsing requirements, thus the parsing time of JSON is estimated to be up to one hundred times quicker than XML [127]. A drawback associated with JSON is that in languages other than Javascript, a custom library is required to parse JSON objects.
YAML
JSON’s strength lies in its simplicity and the facilities it provides to generate, parse and process JSON data across different programming environments. However, this requires JSON to keep structural characters with the data which effects the human readability aspect. To come up with a data serialization language that is truly readable to humans, facilitates data portability across programming languages, is expressive and extensible, YAML (YAML Ain’t Markup Language) [19] was introduced by Clark Evans in 2001.
YAML’s cleanness and delicate variations in grammar helps presenting special cases more precisely. YAML achieves fineness by using very few structural characters, and presents the data in a natural yet meaningful style. Consequently, YAML requires more processing than JSON to port data across programming environments. An interesting fact about YAML is that it is a superset of JSON, and so a valid JSON document can also be viewed as a YAML document [19].
Markdown
Unlike the previous three languages, Markdown is neither a data exchange language, nor it is typically used to format service descriptions. However, one of the emerging service description formats, API Blueprint1, uses Markdown to describe its interfaces, so it is interesting to look at the potential benefits of an unusual use case.
Markdown is a markup language with the key design goal as readability – that means a Markdown document is aimed to be readable as-is, as plain text, without looking like it has been formatted with structural characters or tags [76]. Inspired by WYSIWYG (what you see is what you get) formatting, Markdown is mostly used to create read-me files, and to format rich text using a plain text editor. Markdown can be converted to HTML, XML, JSON, YAML and many other formats.
Among the popular description formats, API Blueprint is the only one based on Markdown. Initially, API Blueprint used JSON to encode data structures, but very recently a new Markdown syntax has been introduced with the name MSON (Mark-down Syntax for Object Notation), which is now used to describe data structures in API Blueprint2. MSON is compatible with describing JSON and JSON Schema, while the support of XML and YAML is aimed for future.
Readability Aspects
All the service description languages, we reviewed in the previous section, are both machine and human readable. Here we elucidate how friendly, in terms of readability, are theses languages for machines and humans.
One of the design goals of XML, according to W3C, is: « XML documents should be human-legible and reasonably clear » [26]. Though XML is text based (Unicoded), but is not considered as a true human-readable because of its random grammar and extra syntax with no real semantics [88].
Unlike XML, JSON does not carry extra baggage of information, and by default matches the data structures of the most of the modern programming languages. The lack of extra tags, smaller grammar and similarities with data structures makes JSON more readable to humans than XML [46].
YAML came up with a design principle of no markups or extra structural informa-tion, and presented an excellent easy-to-read format while keeping the data exchange properties. Markdown took the service description to another level by proposing a documentation-oriented description with WYSIWYG (what you see is what you get) formatting, and left all the heavy-lifting for machines instead of humans. To compare the readability of YAML and Markdown, Figure 3.3 presents an identical API described in YAML (Swagger 2.0) and Markdown (API Blueprint).
To conclude, we found that the earlier breed of service description languages were more machine friendly. They contained extra bits of structural characters, which makes them easier to parse and perform data exchange for machine, but difficult to read for human. That has been one of the reasons for publishing web service documentation in plain natural language along with the standard service description. In contrast, the later languages focused more on readability and easiness of use, which makes them more human friendly.
Emerging Formats
Table 3.1 lists the emerging description formats for service interfaces, which we are going to review in this section. Most of these formats are introduced in the last five years, and are used extensively in industrial applications. The list also includes WSDL, which is not an emerging format, but has ruled the service oriented domain for over a decade, and is still having its roots in many enterprise-level service architectures as well as in academic research.
WSDL 1.1 and 2.0
Both WSDL 1.1 and WSDL 2.0 can technically be used to describe the new types of web services, including RESTful services (though the support is limited in WSDL 1.1). However, the association with SOAP and the lack of RESTfulness, proved to big hurdles for WSDL’s adoption for the modern service architectures. WSDL 2.0 though, technically support the description of REST services, but its core model is not around resources and representations, and the tool-support is not as good as the other formats in the eco system.
WADL
WADL (Web Application Description Language) is an XML based format, that sup-ports both JSON and XML schema for describing resource representations. WADL was one of the earliest attempts to develop a description format for RESTful services with the goal to provide the same functionality for REST that WSDL provides for SOAP. Sun Microsystems submitted WADL specification to the W3C in 2009, but it has not been standardized yet, mainly due to the lack of adoption.
Swagger 1.x and 2.0
Probably the most mature API description format for non-SOAP based services is Swagger, from Wordnik (Wordnik, 2009). Swagger 1.x uses a simple JSON syntax to describe the web interfaces in terms of HTTP methods, media types, data models and status codes. Swagger is mostly used for its interactive documentation feature, and is well-supported by the open source community. Swagger 2.0, released in late 2014, has adopted YAML format instead of JSON for editing, allows third party extensions to its standard specification, and provides more intuitive tooling such as an API console and an API designer3. Swagger also rudimentarily supports client and server code generation in various languages.
API Blueprint
API Blueprint, introduced by Apiary in the first quarter of 2013, is designed specifically to build and document an interface description. By using a documentation-oriented de-scription, API Blueprint eliminates the need of the adscititious service documentation in natural language. However, this high level of readability requires a quite complex parsing of the service description, which is being provided by a good tool support from API Blueprint community4. API Blueprint is the only description format, based on Markdown, and so depends on a JSON like language for data exchange. The reason of choosing Markdown is stated as « API Blueprint is a documentation-oriented API description language… It is easy to learn and even easier to read – after all it is just a form of plain text. »5
RAML
RAML 0.8 was released in the last quarter of 2013 by Mulesoft, while the version 1.0 has also been released in November 2015. RAML’s speciality lies in the fact that it is designed to model an API, not just document it. Like other formats, RAML also comes with powerful supporting tools including an API console, an API Designer, and API Notebook6 – a tool to help developers interact with an API. Like Swagger, RAML is also written in the YAML format, and targets technical as well as non-technical people to read, write and edit an interface description.
ioDocs
ioDocs is among the least popular description formats in our list. Introduced by Mash-ery for REST metadata and documentation, ioDocs uses JSON-formatted metadata to generate an interactive documentation for the described API that can be used to both learn about an API and execute ad-hoc calls against it. In contrast to the other formats described above, ioDocs also includes basic support for signatures and different versions of OAuth (an authentication type, we will cover in the next chapter).
APIMATIC
APIMATIC is a commercial tool (co-founded by the thesis author) to automatically generate [6] client code for APIs in various programming languages. APIMATIC’s import feature can take in any of the discussed description formats and convert it in to its internal SDL (Service Description Language) format, known as APIMATIC format. Because of its universal import capability, APIMATIC format can be seen as a super-set of the all the description formats discussed here. APIMATIC format also supports additional elements7, exclusive to code-generation settings.
Closing Remarks
As the API design world continues to evolve, a number of incentives are emerging for API providers to generate machine readable API descriptions, including interac-tive documentation, client and server side code generation, service mocking, testing and monitoring, discovery, search, reusability, interoperability and copyright, besides specifying a contract. These are the incentives that live throughout the API lifecycle. As a result, a good number of description formats have emerged in recent years, with distinct and overlapping purposes.
We are observing that the API descriptions formats are becoming really the “glue” to the entire API tools ecosystem. It strikes us that a robust contract first API design and development is the way forward to reduce the cruftiness and crafty-ness of the APIs we build.
Contents
1 Introduction
1.1 Web Service Components
1.2 The Central Topic – Service Interface Descriptions (APIs)
1.3 The Evolution of Service Interface Descriptions
1.4 Structural Analysis of Service Interface Descriptions
1.5 Behavioral Modeling from Stateful Service Interfaces
1.6 Web Service Composition
1.7 Thesis Organization
1.8 Contributions – A Bird’s Eye View
2 Literature Review
2.1 Service Interface (API)
2.2 Formal Behavioral Modeling
2.3 Service Composition
2.4 Data Mediation
2.5 Mobile Service Composition
2.6 Closing Remarks
I Service Interface Descriptions – Evolution and Structural Analysis
3 The Evolution of Interface Description Formats
3.1 Introduction
3.2 The Evolution
3.3 Closing Remarks
4 Structural Analysis of APIs, Emerging Trends And Recommendations
4.1 Introduction
4.2 Research Setup
4.3 Analyzing the Interface Descriptions Data
4.4 Answers and Recommendations
4.5 Case Study
4.6 Related Work and Discussion
4.7 Concluding Remarks
II Model Driven Engineering via Service Interfaces
5 Behavioral Modeling from Stateful Service Interfaces
5.1 Introduction
5.2 Illustrative Example
5.3 WSDL to SKS Conversion Methodology
5.4 Compatibility and Limitations
5.5 Evaluation
5.6 Closing Remarks
6 Web Service Composition – A Unified Approach using Interface Descriptions
6.1 Introduction
6.2 Illustrative Example
6.3 Composition Challenges
6.4 Composition Framework
6.5 Composition Demonstration
6.6 Case Study – My Heart Care
6.7 Experimental Evaluation
6.8 Related Work and Discussion
6.9 Closing Remarks
7 Conclusions
7.1 Synopsis
7.2 Conclusions And Future Directions
7.3 Final Words
Bibliography
GET THE COMPLETE PROJECT
Web Service Interfaces – Design, Trends and Model Driven Engineering
