Designing an FPGA-based graphics controller

Dominik Domanski, MYLIUM 1/15/2011 1:35 PM EST

A picture is worth a thousand words. We have all heard this saying many times and we observe the usage of icons, graphic images, pictograms, and pictures everywhere. Electronics is no exception. I remember my first projects where I used a light-emitting diode (LED) to indicate a Logic 1 value on an output pin. After that came 7-segment displays; still later were liquid crystal displays (LCDs) with 2x16 characters (that was a big step at the time...). Some of us even played with simple black-and-white graphic LCDs. When working with these small, low-resolution devices, the task of switching individual pixels On and Off were relatively easy to perform using cheap-and-cheerful low-cost microcontrollers. Many designs were created in this way in fact many are still working around us to this day. However, problems start when we wish to work with something bigger and in color like an LCD monitor. These are relatively cheap and, in many cases, they are already on our desks. Lets assume that were working with a resolution of 800 x 600 = 480,000 pixels, a color depth of 16 bits = 65,535 colors per pixel, and a refresh rate of 50 or 60 Hz. Thus, we will require 960 KB of memory for a single frame to be read 60 times per second. Based on the timing for this resolution, we will need to send data to the display with a frequency of 40 MHz. It would be really hard to transfer this amount of data with a low-end microcontroller while still leaving sufficient computational resources and bandwidth to perform other tasks. Another problem is the lack of free pins (~20 for the external memory plus 10 or more for the display). We can of course take a larger, faster microcontroller with a bunch of free pins, buy new programming tools for it, add some memory to store the frames to be displayed, and rewrite our software. But this solution has some disadvantages though, including being more expensive and requiring us to learn how to use new tools and work with a new microcontroller. An alternative solution is to use an external graphics controller to perform all tasks related to graphics memory and display refresh operations, thereby freeing up our low-cost host microcontroller to perform other operations. An example of such a device is the LAVA 10 SVGA Graphics Controller, which is based on a Spartan 3 XC3S200 FPGA from Xilinx.

Block diagram of the LAVA 10 SVGA Graphics Controller Before starting work on this design we set the following requirements:

A minimum resolution of 800 x 600 (to work with most LCD monitors) The ability to refresh the display 60 times per second

The ability to store and display more than one frame Simple access to external memory (for the host microcontroller) Non-volatile memory (for graphic data, icons etc.) An embedded character generator A simple interface for the host microcontroller

The main difficulty was the data transmission from/to memory. For the display refresh alone we require a bandwidth of 57.6 MB/s. But we also need to write data to memory to modify pixels (drawing) at a reasonable rate, so we decided to increase the speed of the internal logic to 80 MHz. Working with 32 MB of 16-bit wide on-board SDRAM memory gives a theoretical bandwidth of 160 MB/s. This amount of memory is sufficient to store more than one frame; currently we store two frames in it. This way we can display one frame while making changes to the other, and then switch between them (this technique prevents flickering on the screen). The rest of memory can be used by the user to store calculation results, graphics, etc. In most designs we need a non-volatile memory to store graphic files like icons, bitmaps, additional fonts, etc. We decided that our memory should be big enough to be able to store at least one full-screen bitmap and also fast enough to not slow down whole system. Thus, we used 8 MB of 16-bit wide Flash memory with a parallel interface. With an access time of 70 ns this gives a bandwidth of ~28 MB/s, which is much more than SPI Flash memories. To keep pin usage at a reasonable level, both memories share pins. A separate arbiter is used to handle the case in which both memories are trying to be accessed at the same time; for example, reading from the Flash memory while the display is being refreshed (in this case the SDRAM memory will have the highest priority). The next step was choosing the interface with the host microcontroller. Since we intend our device to be universal, this interface needs to be simple and fast. Thus, SPI/I2C interfaces were excluded due to insufficient speed. Modifying an individual pixel requires the transmission of 36 bits (10 bits each for the X and Y coordinates and 16 bits for the new color value). Transmitting this data serially would require 36 clock cycles. A parallel interface would seem to be more natural the best one would be 16-bit wide (because each pixel's value is 16 bits), but this would require ~20 pins on our microcontroller. Therefore, we decided to reduce the size and to use an 8-bit wide bidirectional interface. This way we have 8X the speed of a 1-bit serial interface; also the implementation of such an 8-bit interface is very easy in any microprocessor (reading/writing individual bytes from/to a port). Drawing any character is fairly easy, especially when that character is stored in the Flash memory. To display the letter A, for example, we need only to copy this letter from the Flash memory to SDRAM. However, for many applications we don't require a complex font like Arial in five different sizes; instead, a simple (console-like) font may be sufficient. It is for this reason that such a font can be displayed using our embedded character generator, which has a separate RAM to store characters of 8x16 pixels. Displaying a single frame is consists of writing a single line every ~26 us and 600 lines every ~16 ms. In order to keep this task as a separate process, the Display Controller is equipped with a small FIFO that can store sufficient data to display 1 line (800 pixels). The Display Controller also takes care of maintaining the horizontal and vertical synchronization. To keep all of these processes under control, we created a separate block which we called the 2D Engine to supervise the passing data between memories, from the Memory Controller to the Display Controller, and to handle commands coming from the host microcontroller. The following commands are currently supported:

Draw text (using the embedded character controller)

Read memory (SDRAM/FLASH) Write memory (SDRAM/FLASH) Copy memory (SDRAM->SDRAM, FLASH->SDRAM)

Although these commands may seem to be very basic, they are sufficient for most of tasks like drawing figures, animations etc. Examples can be seen on video clips on YouTube (Click Here to see the first example and Click Here to see the second example). The best thing about basing our Graphics Controller on an FPGA is that we can change and expand the functionality of the controller. Our current design consumes about 60% of the resources of the XC3S200 FPGA, thereby leaving sufficient resources to implement additional functionality. For example, we could add hardware acceleration for tasks like drawing lines, or add some functionality that is not related to the graphics.

Evaluation board with the LAVA 10 module in the middle (JTAG is connected to the top right corner; the small red board on the left is the host microcontroller [a STM32F103 in this case]). Furthermore, we can update our board by simply uploading a bitstream file to it with the latest version of the design. Another possibility is to use our board for creating and testing your own projects; as long as the configuration memory isnt overwritten, you can treat it as any other development kit. Currently, depending on demand, we are planning to improve the uploading of data to the Flash memory and adding a possibility of using external sources like an MMC card. Also adding a simple embedded processor might be an interesting idea in certain applications. For more information, questions, or suggestions please contact me directly as follows: