BAYER TO RGB Bayer-mapped to RGB converter Rev. 1.4 Key Design Features Block Diagram Synthesizable, technology independent VHDL IP Core Decoding of a bayer-mapped image from an image sensor or Colour Filter Array (CFA) 16 16 Supports all image resolutions from 16 x 16 pixels up to 2 x 2 Support for all sensor pixel widths from 2-bits and above Support for different CFA sensor alignments Fully pipelined architecture with simple flow control Input and output ports can interface directly to a FIFO if required Features a 5x5 polyphase interpolation filter Output 1 x 24-bit pixel per clock No frame buffer required Small implementation size 1 Support for 300 MHz+ operation on basic FPGA devices Figure 1: Bayer-to-RGB converter architecture Applications Pin-out Description Bayer-mapped to 24-bit RGB decoding (de-mosaicing) Pin name I/O Description Active state clk in Synchronous clock rising edge Digital camera image processing reset in Asynchronous reset low Forms an essential first stage in most digital processing pixels per line in Number of pixels per line data pipelines that contain an image sensor 4 16 Range: 2 < lines < 2 lines per frame in Number of lines per frame data 4 16 Generic Parameters Range: 2 < lines < 2 pixin dw - 1:0 in n-bit bayer-mapped pixel in data Generic name Description Type Valid range pixin vsync in Vertical sync in high (Coincident with first pixel dw Sensor width in bits integer 2 of input frame) 4 16 line width Width of linestores in integer 2 < pixels < 2 pixin hsync in Horizontal sync in high pixels (Coincident with first pixel of input line) log2 line width Log2 of linestore width integer log2(line width) pixin val in Input pixel valid high sensor align Bayer pattern sensor integer 0 : BGBG ... alignment GRGR pixin rdy out Ready to accept input pixel high (Handshake signal) 1 : GBGB ... RGRG ... pixout 23:0 out 24-bit RGB pixel out data pixout vsync out Vertical sync out high 2 : GRGR ... (Coincident with first pixel BGBG of output frame) 3 : RGRG ... pixout hsync out Horizontal sync out high GBGB ... (Coincident with first pixel of output line) pixout val out Output pixel valid high pixout rdy in Ready to accept output high pixel (Handshake signal) 1 Xilinx Virtex6 used as a benchmark Copyright 2015 www.zipcores.com Download this VHDL Core Page 1 of 4BAYER TO RGB Bayer-mapped to RGB converter Rev. 1.4 Image resolution General Description The size of the image to be interpolated is fully programmable and is BAYER TO RGB (Figure 1) is a fully pipelined Bayer-mapped to RGB specified in the parameters: pixels per line and lines per frame. These converter IP core. The IP core may be used to process the raw pixels parameters can be changed on a frame-by-frame basis if necessary. It is from an image sensor or Colour Filter Array (CFA). These pixels are recommended that a system reset is asserted once the parameters have typically organized as a bayer pattern of discrete Red, Green and Blue been changed to avoid possible image corruption. After reset, the IP core values which must be interpolated to recover the original image - a will start generating output pixels after the next clean input frame. process that is commonly known as de-mosaicing. The generic parameters line width and log2 line width must also be set Internally, the circuit uses a 5x5 pixel filter with dynamic coefficients to correctly to accommodate the maximum line length of the input image. interpolate the pixels from the CFA. The resulting output is a high-quality The line width must be specified as the nearest power of 2 - e.g. 1024, RGB image at 24-bits/pixel. 2048, 4096 etc. Bayer-mapped pixels flow into the design in accordance with the valid 2 ready pipeline protocol . Input pixels and syncs are sampled on the rising De-mosaicing filter edge of clk when pixin val and pixin rdy are both high. Likewise, at the output interface, pixels and syncs are sampled on a the rising edge of clk when pixout val and pixout rdy are both high. The internal filter is a 5x5 pixel filter that is used to interpolate the bayer- mapped image. The filter architecture uses a polyphase filter design in The image size is fully programmable with standard support for anything which the filter kernel changes depending on the interpolation point in the from 16x16 pixels and above. The width of the input pixels is specified bayer pattern. The filter kernels are based on the Malvar-He-Cutler using the generic parameter dw. Output pixels are standard 24-bit RGB. algorithm which has been shown to give excellent results for a very reasonable resource cost. For optimum results it is recommended that the image sensor has a capacity of at least 5M pixels or better. Sensor alignment Functional Timing The sensor alignment parameter modifies the central starting position of the 5x5 filter according to the alignment of the bayer pattern. Figure 2 below demonstrates the 4 possible sensor alignments in the CFA. Figure 3 shows the signalling at the input interface at the start of a new frame. The first line of a new frame begins with pixin vsync and pixin hsync asserted high together with the first pixel. Note that the signals pixin, pixin vsync and pixin hsync are only valid if pixin val is also asserted high. For demonstration purposes, the diagram also shows what happens when pixin rdy is de-asserted. In this case, the pipeline is stalled and the upstream interface must hold-off before further pixels are processed. Figure 3: Start of new input frame Figure 2: CFA sensor alignments Figure 4 shows the signalling at the start of a new line. Note that the By setting the sensor align parameter correctly, the design can adapt to timing diagram is the same as for the start of a new frame with the the four possible patterns. If the alignment is wrong, then the colours in exception that pixin vsync is held low while pixin hsync is held high the output image will be corrupted. together with the first valid pixel. In this example, pixin rdy is held high for the duration so the input interface does not stall. 2 See Zipcores application note: app note zc001.pdf for more examples of how to use the valid-ready pipeline protocol Copyright 2015 www.zipcores.com Download this VHDL Core Page 2 of 4