API Design for C++

world.

Chapter 1

Introduction

Publisher Summary

This chapter defines an Application Programming Interface (API), which provides an abstraction for a problem and specifies how clients should interact with software components that implement a solution to that problem. API development is ubiquitous in modern software development. Its purpose is to provide a logical interface to the functionality of a component while also hiding any implementation details. It can be as small as a single function or involve hundreds of classes, methods, free functions, data types, enumerations, and constants. Its implementation can be proprietary or open source. The important underlying concept is that an API is a well-defined interface that provides a specific service to other pieces of software. An API is an interface designed for developers, in much the same way that a Graphical User Interface (GUI) is an interface designed for end users.

Defines what an Application Programming Interface (API) is with various examples and describes how an API is represented in C++. This chapter details the difference between API development and standard application development, as well as enumerating the pros and cons of APIs in software development. I then illustrate the various layers of APIs that a modern application is built on, with specific reference to a large open source client/server program. The relationship of the terms API and SDK is explained, as is the relationship among APIs, file formats, and network protocols.

1.1 What are Application Programming Interfaces?

An Application Programming Interface (API) provides an abstraction for a problem and specifies how clients should interact with software components that implement a solution to that problem. The components themselves are typically distributed as a software library, allowing them to be used in multiple applications. In essence, APIs define reusable building blocks that allow modular pieces of functionality to be incorporated into end-user applications.

An API can be written for yourself, for other engineers in your organization, or for the development community at large. It can be as small as a single function or involve hundreds of classes, methods, free functions, data types, enumerations, and constants. Its implementation can be proprietary or open source. The important underlying concept is that an API is a well-defined interface that provides a specific service to other pieces of software.

A modern application is typically built on top of many APIs, where some of these can also depend on further APIs. This is illustrated in Figure 1.1, which shows an example application that depends directly on the API for three libraries (1–3), where two of those APIs depend on the API for a further two libraries (4 and 5). For instance, an image viewing application may use an API for loading GIF images, and that API may itself be built upon a lower-level API for compressing and decompressing data.

Figure 1.1 An application that calls routines from a hierarchy of APIs. Each box represents a software library where the dark section represents the public interface, or API, for that library, while the white section represents the hidden implementation behind that API.

API development is ubiquitous in modern software development. Its purpose is to provide a logical interface to the functionality of a component while also hiding any implementation details. For example, our API for loading GIF images may simply provide a LoadImage() method that accepts a filename and returns a 2D array of pixels. All of the file format and data compression details are hidden behind this simple interface. This concept is also illustrated in Figure 1.1, where client code only accesses an API via its public interface, shown as the dark section at the top of each box.

1.1.1 Contracts and Contractors

As an analogy, consider the task of building your own home. If you were to build a house entirely on your own, you would need to possess a thorough understanding of architecture, plumbing, electronics, carpentry, masonry, and many other trades. You would also need to perform every task yourself and keep track of the minutest of details for every aspect of the project, such as whether you have enough wood for your floorboards or whether you have the right fasteners to fit the screws that you've bought. Finally, because you are the only person working on the project, you can only perform a single task at any point in time and hence the total time to complete the project could be very large.

An alternative strategy is to hire professional contractors to perform key tasks for you (Figure 1.2). You could hire an architect to design the plans for the house, a carpenter for all of your woodwork needs, a plumber to install the water pipes and sewage system for your house, and an electrician to set up the power systems. Taking this approach, you negotiate a contract with each of your contractors—telling them what work you want done and agreeing upon a price—they then perform that work for you. If you're lucky, maybe you even have a good friend who is a contractor and he offers you his services for free. With this strategy, you are freed from the need to know everything about all aspects of building a house and instead you can take a higher-level supervisory role to select the best contractors for your purpose and ensure that the work of each individual contractor is assembled together to produce the vision of your ideal home.

Figure 1.2 Using contractors to perform specialized tasks to build a house.

The analogy to APIs is probably obvious: the house that you're building equates to a software program that you want to write, and the contractors provide APIs that abstract each of the tasks you need to perform and hide the implementation details of the work involved. Your task then resolves to selecting the appropriate APIs for your application and integrating them into your software. The analogy of having skilled friends who provide contracting services for free is meant to represent the use of freely available open source libraries in contrast to commercial libraries that require a licensing fee to use in your software. The analogy could even be extended by having some of the contractors employing subcontractors, which corresponds to certain APIs depending on other APIs to perform their task.

The contractor analogy is a common one in object-oriented programming. Early practitioners in the field talked about an object defining a binding contract for the services or behavior that it provides. An object then implements those services when asked to by a client program, potentially by subcontracting some of the work out to other objects behind the scenes (Meyer, 1987; Snyder, 1986).

1.1.2 APIs in C++

Strictly speaking, an API is simply a description of how to interact with a component. That is, it provides an abstraction and a functional specification for a component. In fact, many software engineers prefer to expand the acronym API as Abstract Programming Interface instead of Application Programming Interface.

In C++, this is embodied as one or more header (.h) files plus supporting documentation files. An implementation for a given API is often represented as a library file that can be linked into end-user applications. This can be either a static library, such as a .lib file on Windows or .a on Mac OS X and Linux, or a dynamic library such as a .dll file on Windows, .dylib on Mac, or .so on Linux.

A C++ API will therefore generally include the following elements:

1. Headers: A collection of .h header files that define the interface and allow client code to be compiled against that interface. Open source APIs also include the source code (.cpp files) for the API implementation.

2. Libraries: One or more static or dynamic library files that provide an implementation for the API. Clients can link their code against these library files in order to add the functionality to their applications.

3. Documentation: Overview information that describes how to use the API, often including automatically generated documentation for all of the classes and functions in the API.

As an example of a well-known API, Microsoft's Windows API (often referred to as the Win32 API) is a collection of C functions, data types, and constants that enable programmers to write applications that run on the Windows platform. This includes functions for file handling, process and thread management, creating graphical user interfaces, talking to networks, and so on.

The Win32 API is an example of a plain C API rather than a C++ API. While you can use a C API directly from a C++ program, a good example of a specific C++ API is the Standard Template Library (STL). The STL contains a set of container classes, iterators for navigating over the elements in those containers, and various algorithms that act on those containers (Josuttis, 1999). For instance, the collection of algorithms includes high-level operations such as std::search(), std::reverse(), std::sort(), and std::set_intersection(). The STL therefore presents a logical interface to the task of manipulating collections of elements, without exposing any of the internal details for how each algorithm is implemented.

Tip

An API is a logical interface to a software component that hides the internal details required to implement it.

1.2 What's Different About API Design?

Interfaces are the most important code that a developer writes. That's because problems in an interface are far more costly to fix than problems in the associated implementation code. As a result, the process of developing shared APIs demands more attention than standard application or Graphical User Interface (GUI) development. Of course, both should involve best design practices; however, in the case of API development, these are absolutely critical to its success. Specifically, some of the key differentiating factors of API development include the following.

• An API is an interface designed for developers, in much the same way that a GUI is an interface designed for end users. In fact, it's been said that an API is a user interface for programmers (Arnold, 2005). As such, your API could be used by thousands of developers around the world, and it will undoubtedly be used in ways that you never intended (Tulach, 2008). You must anticipate this in your design. A well-designed API can be your organization's biggest asset. Conversely, a poor API can create a support nightmare and even turn your users toward your competitors (Bloch, 2005), just as a buggy or difficult-to-use GUI may force an end user to switch to a different application.

• Multiple applications can share the same API. Figure 1.1 showed that a single application can be composed of multiple APIs. However, any one of those APIs could also be reused in several other applications. This means that while problems in the code for any given application will only affect that one application, errors in an API can affect all of the applications that depend on that functionality.

• You must strive for backward compatibility whenever you change an API. If you make an incompatible change to your interface, your clients' code may fail to compile, or worse their code could compile but behave differently or crash intermittently. Imagine the confusion and chaos that would arise if the signature of the printf() function in the standard C library was different for different compilers or platforms. The simple Hello World program may not look so simple any more:

#include

#ifdef _WIN32

#include

#endif

#ifdef __cplusplus

#include

#endif

int main(int argc, char *argv[])

{

#if defined(__STRICT_ANSI__)

printf(Hello World\n);

#elif defined(_WIN32)

PrintWithFormat(Hello World\n);

#elif defined(__PRINTF_DEPRECATED__)

fprintf(stdout, Hello World\n);

#elif defined(__PRINTF_VECTOR__)

const char *lines[2] = {Hello World, NULL};

printf(lines);

#elif defined(__cplusplus)

std::cout << Hello World << std::endl;

#else

#error No terminal output API found

#endif

return 0;

}

This may seem like a contrived example, but it's actually not that extreme. Take a look at the standard header files that come with your compiler and you will find declarations that are just as convoluted and inscrutable, or perhaps worse.

• Due to the backward compatibility requirement, it's critical to have a change control process in place. During the normal development process, many developers may fix bugs or add new features to an API. Some of these developers may be junior engineers who do not fully understand all of the aspects of good API design. As a result, it's important to hold an API review before releasing a new version of the API. This involves one or more senior engineers checking that all changes to the interface are acceptable, have been made for a valid reason, and are implemented in the best way to maintain backward compatibility. Many open source APIs also enforce a change request process to gain approval for a change before it is added to the source code.

• APIs tend to live for a long time. There can be a large upfront cost to produce a good API because of the extra overhead of planning, design, versioning, and review that's necessary. However, if done well, the long-term cost can be substantially mitigated because you have the ability to make radical changes and improvements to your software without disrupting your clients. That is, your development velocity can be greater due to the increased flexibility that the API affords you.

• The need for good documentation is paramount when writing an API, particularly if you do not provide the source code for your implementation. Users can look at your header files to glean how to use it, but this does not define the behavior of the API, such as acceptable input values or error conditions. Well-written, consistent, and extensive documentation is therefore an imperative for any good API.

• The need for automated testing is similarly very high. Of course, you should always test your code, but when you're writing an API you may have hundreds of other developers, and thousands of their users, depending on the correctness of your code. If you are making major changes to the implementation of your API, you can be more confident that you will not break your clients' programs if you have a thorough suite of regression tests to verify that the desired API behavior has not changed.

Writing good APIs is difficult. While the necessary skills are founded on general software design principles, they also require additional knowledge and development processes to address the points just listed. However, the principles and techniques of API design are rarely taught to engineers. Normally, these skills are only gained through experience—by making mistakes and learning empirically what does and does not work (Henning, 2009). This book is an attempt to redress this situation, to distill the strategies of industrial-strength, future-proof API design that have been evolved through years of software engineering experience into a comprehensive, practical, and accessible format.

Tip

An API describes software used by other engineers to build their applications. As such, it must be well-designed, documented, regression tested, and stable between releases.

1.3 Why Should you Use APIs?

The question of why you should care about APIs in your own software projects can be interpreted in two different ways: (1) why should you design and write your own APIs or (2) why should you use APIs from other providers in your applications? Both of these perspectives are tackled in the following sections as I present the various benefits of using APIs in your projects.

1.3.1 More Robust Code

If you are writing a module to be used by other developers, either for fellow engineers within your organization or for external customers of your library, then it would be a wise investment to create an API for them to access your functionality. Doing so will offer you the following benefits.

• Hides implementation. By hiding the implementation details of your module, you gain the flexibility to change the implementation at a future date without causing upheaval for your users. Without doing so, you will either (i) restrict yourself in terms of the updates you can make to your code or (ii) force your users to rewrite their code in order to adopt new versions of your library. If you make it too onerous for your clients to update to new versions of your software, then it's highly likely that they will either not upgrade at all or look elsewhere for an API that will not be as much work for them to maintain. Good API design can therefore significantly affect the success of your business or project.

• Increases longevity. Over time, systems that expose their implementation details tend to devolve into spaghetti code where every part of the system depends on the internal details of other parts of the system. As a result, the system becomes fragile, rigid, immobile, and viscous (Martin, 2000). This often results in organizations having to spend significant effort and money to evolve the code toward a better design or simply rewrite it from scratch. By investing in good API design up front and paying the incremental cost to maintain a coherent design, your software can survive for longer and cost less to maintain in the long run. I'll delve much deeper into this point at the start of Chapter 4.

• Promotes modularization. An API is normally devised to address a specific task or use case. As such, APIs tend to define a modular grouping of functionality with a coherent focus. Developing an application on top of a collection of APIs promotes loosely coupled and modular architectures where the behavior of one module is not dependent on the internal details of another module.

• Reduces code duplication. Code duplication is one of the cardinal sins of software engineering and should be stamped out whenever possible. By keeping all of your code's logic behind a strict interface that all clients must use, you centralize the behavior in a single place. Doing so means that you have to update only one place to change the behavior of your API for all of your clients. This can help remove duplication of implementation code throughout your code base. In fact, many APIs are created after discovering duplicated code and deciding to consolidate it behind a single interface. This is a good thing.

• Removes hardcoded assumptions. Many programs may contain hardcoded values that are copied throughout the code, for example, using the filename myprogram.log whenever data are written to a log file. Instead, APIs can be used to provide access to this information without replicating these constant values across the code base. For example, a GetLogFilename() API call could be used to replace the hardcoded myprogram.log string.

• Easier to change the implementation. If you have hidden all of the implementation details of your module behind its public interface then you can change those implementation details without affecting any code that depends on the API. For example, you might decide to change a file parsing routine to use std::string containers instead of allocating, freeing, and reallocating your own char * buffers.

• Easier to optimize. Similarly, with your implementation details hidden successfully, you can optimize the performance of your API without requiring any changes to your clients' code. For example, you could add a caching solution to a method that performs some computationally intensive calculation. This is possible because all attempts to read and write your underlying data are performed via your API, so it becomes much easier to know when you must invalidate your cached result and recompute the new value.

1.3.2 Code Reuse

Code reuse is the use of existing software to build new software. It is one of the holy grails of modern software development. APIs provide a mechanism to enable code reuse.

In the early years of software development, it was common for a company to have to write all of the code for any application they produced. If the program needed to read GIF images or parse a text file, the company would have to write all that code in-house. Nowadays, with the proliferation of good commercial and open source libraries, it makes much more sense to simply reuse code that others have written. For example, there are various open source image reading APIs and XML parsing APIs that you can download and use in your application today. These libraries have been refined and debugged by many developers around the world and have been battle-tested in many other programs.

In essence, software development has become much more modular, with the use of distinct components that form the building blocks of an application and talk together via their published APIs. The benefit of this approach is that you don't need to understand every detail of every software component, in the same way that for the earlier house building analogy you can delegate many details to professional contractors. This can translate into faster development cycles, either because you can reuse existing code or decouple the schedule for various components. It also allows you to concentrate on your core business logic instead of having to spend time reinventing the wheel.

One of the difficulties in achieving code reuse, however, is that you often have to come up with a more general interface than you originally intended. That's because other clients may have additional expectations or requirements. Effective code reuse therefore follows from a deep understanding of the clients of your software and designing a system that integrates their collective interests with your own.

C++ APIs and the Web

The trend toward applications that depend on third-party APIs is particularly popular in the field of cloud computing. Here, Web applications rely more and more on Web services (APIs) to provide core functionality. In the case of Web mashups, the application itself is sometimes simply a repackaging of multiple existing services to provide a new service, such as combining the Google Maps API with a local crimes statistics database to provide a map-based interface to the crime data.

In fact, it's worth taking a few moments to highlight the importance of C++ API design in Web development. A superficial analysis might conclude that server-side Web development is confined to scripting languages, such as PHP, Perl, or Python (the P in the popular LAMP acronym), or .NET languages based on Microsoft's ASP (Active Server Pages) technology. This may be true for small-scale Web development. However, it is noteworthy that many large-scale Web services use a C++ backend to deliver optimal performance.

In fact, Facebook developed a product called HipHop to convert their PHP code into C++ to improve the performance of their social networking site. C++ API design therefore does have a role to play in scalable Web service development. Additionally, if you develop your core APIs in C++, not only can they form a high-performance Web service, but your code can also be reused to deliver your product in other forms, such as desktop or mobile phone versions.

As an aside, one potential explanation for this shift in software development strategy is the result of the forces of globalization (Friedman, 2008; Wolf, 2004). In effect, the convergence of the Internet, standard network protocols, and Web technologies has created a leveling of the software playing field. This has enabled companies and individuals all over the world to create, contribute, and compete with large complex software projects. This form of globalization promotes an environment where companies and developers anywhere in the world can forge a livelihood out of developing software subsystems. Other organizations in different parts of the world can then build end-user applications by assembling and augmenting these building blocks to solve specific problems. In terms of our focus here, APIs provide the mechanism to enable this globalization and componentization of modern software development.

1.3.3 Parallel Development

Even if you're writing in-house software, your fellow engineers will very likely need to write code that uses your code. If you use good API design techniques, you can simplify their lives and, by extension, your own (because you won't have to answer as many questions about how your code works or how to use it). This becomes even more important if multiple developers are working in parallel on code that depends upon each other.

For example, let's say that you're working on a string encryption algorithm that another developer wants to use to write data out to a configuration file. One approach would be to have the other developer wait until you're finished with your work and then he can use it in his file writer module. However, a far more efficient use of time would be for the two of you to meet early on and agree upon an appropriate API. Then you can put that API in place with placeholder functionality that your colleague can start calling immediately, such