Ultrasonic array research platform the main controller transmitters

Project Number: 85 Ultrasonic Array Research Platform (UARP) The Main Controller & Transmitters Chau Vo Ky Submitted in accordance with the requirements for the degree of Master of Science (Eng.) In Embedded Systems Engineering The University of Leeds School of Electronic and Electrical Engineering August 2010 Supervisor Dr S Freear The candidate confirms that the work submitted is his/her own and that appropriate credit has been given where reference has been made to the work of others The University of Leeds School of Electronic & Electrical Engineering Declaration of Academic Integrity Student Name: Chau Vo Ky University ID Number: 200510695 Modules: ELEC 5880 & 5881 Module Leader: Dr Chris Trayner Plagiarism in University Assessments and the Presentation of Fraudulent or Fabricated Coursework Plagiarism is defined as presenting someone else’s work as your own Work means any intellectual output, and typically includes text, data, images, sound or performance1 Fraudulent or fabricated coursework is defined as work, particularly reports of laboratory or practical work that is untrue and/or made up, submitted to satisfy the requirements of a University Assessment in whole or in part Declaration:  I have read the University Regulations on Cheating and Plagiarism and state that the work in this report is my own and does not contain any unacknowledged work from other sources  I confirm that any mitigating circumstances or other matters which might have affected my performance and which I wish to bring to the attention of the examiners/markers have been submitted to the Secretary of the Director of Learning & Teaching [Jennifer Finnigan, School of Electronic & Electrical Engineering] Signed ………………………………… Date ………………………… http://www.leeds.ac.uk/AAandR/cpff.htm http://www.leeds.ac.uk/AAandR/cpff.htm http://www.leeds.ac.uk/studenthandbook/ Abstract Ultrasound Array Research Platform (UARP) is a FPGA-based general platform for ultrasonic imaging It supports a 96-channel beamformer for novel research in ultrasound The platform consists of a main controller, twelve receiver boards, twelve transmitter boards, and two backplanes This report focuses on the development of the FPGA-based hardware and its firmware for the controller board and transmitter boards The hardware is basically built around a Nios-II soft processor And the firmware is written in C on Eclipse IDE For the main controller, the work is to build a Nios-based general platform and improve the USB speed between the platform and a Matlab application on PC It uses a DMA controller to free Nios-II from data transfers with the USB controller, i.e ISP1761 Hardware techniques, e.g separate memories, to further increase the system performance are also described For the firmware, a closer look at a third-party USB driver is presented; required modifications and debug are indicated as well The maximum speed is up to 27 MBps, compared to about MBps on the original driver In addition, timing constraints and the distribution of 50 MHz clock and trigger signals from the main controller to twelve transmitter boards are also discussed They are aligned within 200 ps For transmitter design, a method of using PLLs and timing constraints to achieve the accuracy of phase delay of 96 channels is described It is related to alignment and phase shift of channelto-channel and board-to-board outputs The accuracy of alignment is in sub-nanoseconds, and the minimum phase step achieved is 200 picoseconds A conference paper about this method will be published on 2010 IEEE International Ultrasonic Symposiums [36] The work has been integrated with results of colleagues to build a complete UARP system Contents Abstract .3 List of Figures .8 List of Tables 11 Acknowledgement .12 PART I - INTRODUCTION 13 Introduction to Ultrasound Imaging 14 1.1 The Physics of Ultrasound 14 1.2 Transducers 15 1.3 Excitation Methods 15 1.4 Beamforming 17 1.5 Ultrasound Medical Imaging 18 1.5.1 Ultrasound Imaging Systems 19 Ultrasonic Array Research Platform (UARP) Project Overview 20 2.1 Altera DE3 FPGA Controller Board .23 2.2 Transmitter Board .24 2.3 Receiver Board 25 2.4 Backplane 25 2.5 Matlab Application .26 2.6 Division of Labour .26 2.7 Design Tools .28 2.7.1 The Altera Quartus II Design Software 28 2.7.2 SOPC Builder and Nios-II Embedded Design Suite (EDS) 28 2.7.3 TimeQuest Timing Analyzer 29 PART II - DESIGN AND RESULTS 30 The Main Controller Design .31 3.1 Introduction 31 3.2 FPGA-based Hardware Design 32 3.2.1 The Nios-based platform 33 3.2.2 The USB interface 33 3.2.3 The transmitter interface 34 3.2.4 The receiver interface 35 3.3 3.3.1 Application Layer 36 3.3.2 Bulk Driver Layer 37 3.3.3 USB Protocol Layer .37 3.3.4 ISP1761 Device Control Layer .37 3.3.5 ISP1761 HAL .38 3.3.6 Firmware Improvement 39 3.4 Firmware Design 36 Protocol 41 Transmitter Design 43 4.1 Introduction 44 4.2 FPGA-based Hardware Design 46 4.2.1 Generating excitation waveform using DDS blocks 47 4.2.2 4.3 FPGA-based hardware for pulse generation and phase control 48 Phase Alignment of Transmitting Channels .51 4.3.1 Clock signals 51 4.3.2 The delay in paths 52 4.3.3 Constraints 54 4.3.4 Alignment of high voltage outputs 56 4.4 Phase Control of Transmitting Channels .59 4.5 Distribution of Clock and Trigger Signals 61 4.5.1 Distribution of the clock signals from the controller board 61 4.5.2 Distribution of the trigger signals from the controller board 62 4.6 Firmware Design 64 Results .65 Future Works .68 Conclusion 68 References .70 Appendix A .73 Appendix B .78 Appendix C .80 Appendix D .84 Appendix E .87 Appendix F 89 Appendix G .92 Appendix H .94 Appendix I 96 List of Figures Figure Comparison between pulsed and LFM coded excitation systems [14] p 99 16 Figure Coded excitation with complementary Golay codes [12] 16 Figure (a) an unfocused, (b) a mechanical focusing, and (c) an electronic focusing beam [17] p.91 17 Figure Focusing in transmission of a phased array linear transducer [4] p.90 17 Figure Representation of A-mode scan [4] p.87 .18 Figure M-mode scan, the echo strength is displayed as brightness versus time [4] p.87 18 Figure An illustration of B-mode scan [4] p.88 19 Figure Block diagram of a generic digital ultrasound imaging system [3] p.302 19 Figure UARP System Overview .21 Figure 10 Diagram of the controller board and its interfaces .23 Figure 11 Structure of transmitter board 24 Figure 12 Diagram of a receiver board 25 Figure 13 Diagram of the main controller 32 Figure 14 USB interface with and without a DMA controller 33 Figure 15 Structure of the transmitter interface 34 Figure 16 Structure of the Nios-based firmware 36 Figure 17 The application layer 36 Figure 18 Accessing the ISP1761 chip without a DMA controller 38 Figure 19 Accessing the ISP1761 chip with a DMA controller 39 Figure 20 The waveform before and after debugging 40 Figure 21 The USB speed for different sizes of data 41 Figure 22 A comparison of two URB submissions .41 Figure 23 Two 32-bit words sent from the host to the main controller for controlling 42 Figure 24 A comparison of quinary LFM excitation and analogue excitation [14] .44 Figure 25 Schematic of the array for scanning [34] 45 Figure 26 Diagram of a transmitter board in the UARP system 46 Figure 27 Diagram of a MAX4811 pulser device .47 Figure 28 The DDS block pin diagram 47 Figure 29 Structure to generate the excitation waveform and control the phase in a transmitter 48 Figure 30 Structure of a transmitter (TX) board 49 Figure 31 5-level MHz excitation pulse 50 Figure 32 Clock generation .51 Figure 33 Two transmitting channels .52 Figure 34 Two examples of the data path of two FPGA pins .53 Figure 35 The timing report and waveforms on j9 pins 56 Figure 36 The timing report and waveforms on high voltage outputs for 2-level waveform, FOR disabled 57 Figure 37 The timing report and waveforms on high voltage outputs for 2-level waveform, FOR enabled .58 Figure 38 Control inputs, clock input, and clock outputs of a PLL 59 Figure 39 A result of shifting the phase between two outputs 60 Figure 40 Clock and trigger signals from the controller board on a transmitter board 61 Figure 41 The report on clock output ports 62 Figure 42 The report on Trigger output ports .63 Figure 43 The UARP system with a transducer 66 Figure 44 The standard test block .66 Figure 45 Results with 48 channels 67 Figure 46 TimeQuest Analysis [Altera forum] 92 10 if(data_bit & 0x8000) { WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, } else { WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, WR(PIO_ADC_SPI_BASE, } SPI_DATA); SPI_DATA); SPI_DATA | SPI_CLK); SPI_DATA | SPI_CLK); 0); 0); SPI_CLK); SPI_CLK); i++; } // Set CSn high and CLK high WR(PIO_ADC_SPI_BASE, SPI_CSn | SPI_CLK ); } 110 Main controller – main.c #include #include #include #include #include "terasic_includes.h" "terasic_usb_isp1761\usb_device\bulk_device.h" "altera_avalon_spi_32bits.h" #ifdef DEBUG_APP #define DEBUG_OUT(x) #define DEBUG_ERR(x) #else #define DEBUG_OUT(x) #define DEBUG_ERR(x) #endif {printf("[APP]"); printf x;} {printf("[APP_ERR]!!!"); printf x;} // Communications variables, two 32-bit words alt_u32 US_Data; alt_u32 US_Command; alt_u8 alt_u8 alt_u8 alt_u8 alt_32 *pUS_Channel = (char *)&US_Command; *pUS_Command; *pUS_Board; *pUS_Check; adc_en1, adc_en2, adc_en3, adc_en4; // define for USB testing #define OP_LED_SET #define OP_GET_DATA2 #define OP_GET_DATA3 #define OP_GET_DATA4 #define OP_GET_DATA5 #define OP_GET_DATA6 #define OP_GET_ADC // define command for receiver, used with US_Check = 0x55 #define SET_ADC_DELAY 111 #define #define #define #define #define #define SET_ADC_EN_R1 SET_ADC_EN_R2 SET_ADC_EN_R3 SET_ADC_EN_R4 SET_ADC_START RD_DATA_CHANNEL void welcome_led(void){ const alt_u32 led_delay = 500*1000; DEBUG_OUT(("S welcome_led\n")); IOWR(PIO_LED_BASE,0, usleep(led_delay); IOWR(PIO_LED_BASE,0, usleep(led_delay); IOWR(PIO_LED_BASE,0, usleep(led_delay); IOWR(PIO_LED_BASE,0, usleep(led_delay); IOWR(PIO_LED_BASE,0, usleep(led_delay); // 20100201 chau 0xFFFF00); // blue 0xFF00FF); // green 0x00FFFF); // red 0xFFFFFF); // black 0x000000); // while DEBUG_OUT(("E welcome_led\n")); // 20100201 chau } #define DATAIN_LEN // two 32-bit words: US_Command, US_Data #define DATAOUT_LEN 2048 // testing only // USB driver variables alt_u8 szBulkDataIn[DATAIN_LEN]; alt_u8 szBulkDataOut[DATAOUT_LEN]; BULK_HANDLE hBulk = 0; // functions in application layer void WriteData(BULK_HANDLE hBulk, alt_u32 length); // testing only void ReadOneChannel(BULK_HANDLE hBulk, alt_u32 position); void ReadChannels(BULK_HANDLE hBulk, alt_u32 position); void Notify_DeviceReady(void); void Notify_DataIn(alt_u8 *pData, alt_u32 bytes); 112 void Notify_DataOutComplete(void); // 20100202 chau, testing only /* * Assign the first address to pointer p * Write length bytes to ISP1761 */ void WriteData(BULK_HANDLE hBulk, alt_u32 length){ alt_u8* p; DEBUG_OUT(("S WriteData\n")); // 20100201 chau p = (alt_u8*)IP_ADC_CHANNEL_0_BASE; BULK_Write(hBulk, p, length); DEBUG_OUT(("E WriteData\n")); // 20100201 chau } /* * Used with DLL for reading channels, 8KB/channel, 96-channel version * position[31:24]: the beginning channel * position[23:16]: number of channels */ void ReadChannels(BULK_HANDLE hBulk, alt_u32 position){ alt_u8 ch; alt_u8 num; alt_u8* p; ch = (position >> 24) & 0xFF; num = (position >> 16) & 0xFF; p = (alt_u8*)IP_ADC_CHANNEL_0_BASE + ch * 8192; BULK_Write(hBulk, p, num * 8192); DEBUG_OUT(("Read channels\n")); DEBUG_OUT(("ch, value=%d\n", ch)); DEBUG_OUT(("num, value=%d\n", num)); } /* * Used with DLL for reading one channel, 8KB/channel, 96-channel version 113 * position[31:24]: the channel position */ void ReadOneChannel(BULK_HANDLE hBulk, alt_u32 position){ alt_u8 ch; alt_u8* p; ch = (position >> 24) & 0xFF; p = (alt_u8*)IP_ADC_CHANNEL_0_BASE + ch * 8192; BULK_Write(hBulk, p, position); DEBUG_OUT(("Read One Channel\n")); DEBUG_OUT(("ch, value=%d\n", ch)); } /* * USB driver: not modify */ void Notify_DeviceReady(void){ DEBUG_OUT(("S Notify_DeviceReady\n")); DEBUG_OUT(("Device Ready\n")); // 20100201 chau if (!hBulk) return; BULK_Read(hBulk, szBulkDataIn, sizeof(szBulkDataIn)); DEBUG_OUT(("E Notify_DeviceReady\n")); // 20100201 chau } /* * Receive data and process commands */ void Notify_DataIn(alt_u8 *pData, alt_u32 bytes){ DEBUG_OUT(("S Notify_DataIn\n")); // 20100201 chau DEBUG_OUT(("Data In, len=%d\n", (int)bytes)); alt_u32 rxfull1, rxfull2, rxfull3, rxfull4; if (!hBulk) return; if (bytes == DATAIN_LEN){ alt_u8 OP; 114 alt_u32 Command32[2]; alt_u32 *spi_rdData; memcpy(&Command32, pData, sizeof(Command32)); US_Command = Command32[0]; US_Data = Command32[1]; DEBUG_OUT(("US_Command, len=%X\n", US_Command)); DEBUG_OUT(("US_Data, len=%X\n", US_Data)); // Check if comms byte is correct if ( *pUS_Check == 0xAA ) { // Determine which card receives the command switch(*pUS_Board) { // DE3 - Stratix case 0: switch(*pUS_Command) { case : // LEDs Black IOWR(PIO_LED_BASE, 0, US_Data & 0xFFFFFF); break; case : // Pulse all transmitting channels // chau 08/2010: clear EP0TX buffer - avoid 512 dummy bytes HAL1761_RegWrite32(DC_EP_INDEX,EP0TX); HAL1761_RegWrite32(DC_DMA_COMMAND,0x0F); HAL1761_RegWrite32(DC_DMA_COMMAND,0x0F); // start ADC receiver IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x1); 0x1); 0x1); 0x0); case : // Disable pulse devices = Enable n_shdn (NO pulsing allowed) IOWR_ALTERA_AVALON_PIO_DATA(GPO_1_BASE, 0x0); break; 115 case : // Enable pulse devices = Disable n_shdn (pulsing allowed) IOWR_ALTERA_AVALON_PIO_DATA(GPO_1_BASE, 0x1); break; case : // Set ADC delay IOWR(GPO_0_BASE, 0, US_Data); break; case : // Enable channel 23 - IOWR(CHANNEL_RXEN_R1_BASE, 0, US_Data); break; case : // Enable channel 47 - 24 IOWR(CHANNEL_RXEN_R2_BASE, 0, US_Data); break; case : // Enable channel 71 - 48 IOWR(CHANNEL_RXEN_R3_BASE, 0, US_Data); break; case : // Enable channel 95 - 72 IOWR(CHANNEL_RXEN_R4_BASE, 0, US_Data); break; case : // Start ADC break; case 10 : // Read channels while (! IORD(ADC_FULL_BASE, 0)); IOWR(ADC_START_BASE, 0, 0x00000000); ReadChannels(hBulk,US_Data); break; case 11 : // Pulse, start receivers, wait, and receive one channel IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x1); IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x1); IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x0); // Start ADC, after a delay IOWR(ADC_START_BASE, 0, 0x00000001); 116 while (IORD(ADC_FULL_BASE, 0)); // Wait until ADC memory full while(! IORD(ADC_FULL_BASE, 0)); IOWR(ADC_START_BASE, 0, 0x00000000); // Send data via USB! ReadOneChannel(hBulk,US_Data); break; case 12 : // Pulse, start receivers, wait, and receive channels IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x1); IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x1); IOWR_ALTERA_AVALON_PIO_DATA(PIO_PULSE_BASE, 0x0); // Start ADC, after a delay IOWR(ADC_START_BASE, 0, 0x00000001); // Wait until ADC memory full while(! IORD(ADC_FULL_BASE, 0)); IOWR(ADC_START_BASE, 0, 0x00000000); // Send data via USB! ReadChannels(hBulk,US_Data); break; default: break; } break; // Cyclone TX daughter boards case 1: case 2: case 3: case 4: case 5: case 6: case 7: case 8: case 9: case 10: 117 case 11: case 12: alt_avalon_spi_command_32bits(SPI_BASE, *pUS_Board, 1, &Command32[0], 0, spi_rdData, 0x00); usleep(1); alt_avalon_spi_command_32bits(SPI_BASE, *pUS_Board, 1, &Command32[1], 0, spi_rdData, 0x00); break; default: break; }; } // Receiver Testing else if(*pUS_Check == 0x55) { switch(*pUS_Command) { case SET_ADC_DELAY: IOWR(GPO_0_BASE, 0, US_Data); break; case SET_ADC_EN_R1: adc_en1 = US_Data; DEBUG_OUT(("adc enanle %X \n", US_Data)); IOWR(CHANNEL_RXEN_R1_BASE, 0, US_Data); break; case SET_ADC_EN_R2: adc_en2 = US_Data; DEBUG_OUT(("adc enable %X \n", US_Data)); IOWR(CHANNEL_RXEN_R2_BASE, 0, US_Data); break; case SET_ADC_EN_R3: adc_en3 = US_Data; DEBUG_OUT(("adc enable %X \n", US_Data)); IOWR(CHANNEL_RXEN_R3_BASE, 0, US_Data); break; case SET_ADC_EN_R4: adc_en4 = US_Data; DEBUG_OUT(("adc enable %X \n", US_Data)); IOWR(CHANNEL_RXEN_R4_BASE, 0, US_Data); break; case SET_ADC_START: IOWR(ADC_START_BASE, 0, 0x00000001); 118 while (IORD(ADC_FULL_BASE, 0)); DEBUG_OUT("ADC_start \n"); break; case RD_DATA_CHANNEL: DEBUG_OUT(("read channel %X \t\t %X %X %X %X\n", US_Data, rxfull1, rxfull2, rxfull3, rxfull4)); while (! IORD(ADC_FULL_BASE, 0)); rxfull1= IORD(CHANNEL_RXFULL_R1_BASE, 0); rxfull2= IORD(CHANNEL_RXFULL_R2_BASE, 0); rxfull3= IORD(CHANNEL_RXFULL_R3_BASE, 0); rxfull4= IORD(CHANNEL_RXFULL_R4_BASE, 0); DEBUG_OUT(("read channel %X \t\t %X %X %X %X\n", US_Data, rxfull1, rxfull2, rxfull3, rxfull4)); IOWR(ADC_START_BASE, 0, 0x00000000); ReadChannels(hBulk,US_Data); break; default: break; } } else { //Comms check byte is incorrect - Old command structure - USB Testing OP = (Command32[0]) & 0xFF; if (OP == OP_LED_SET){ alt_u32 data; data = Command32[1]; DEBUG_OUT(("%d\n", data)); IOWR(PIO_LED_BASE, 0, Command32[1] & 0xFFFFFF); }else if (OP == OP_GET_DATA2){ WriteData(hBulk,512); }else if (OP == OP_GET_DATA3){ WriteData(hBulk,8*1024); }else if (OP == OP_GET_DATA4){ WriteData(hBulk,64*1024); }else if (OP == OP_GET_DATA5){ WriteData(hBulk,64*1024*1024); }else if (OP == OP_GET_DATA6){ WriteData(hBulk,256*1024*1024); }else if (OP == OP_GET_ADC){ //IOWR(ADC_START_BASE, 0, 1); 119 //usleep(100); while( !(IORD(ADC_FULL_BASE, 0) & 0x01) ) //IOWR(ADC_START_BASE, 0, 0); WriteData(hBulk,64*1024); }else{ DEBUG_OUT(("Invalid OP\n")); } } } BULK_Read(hBulk, szBulkDataIn, sizeof(szBulkDataIn)); DEBUG_OUT(("E Notify_DataIn\n")); // issue read urb for a next read command // 20100201 chau } /* * USB driver: not modify */ void Notify_DataOutComplete(void){ DEBUG_OUT(("S Notify_DataOutComplete\n")); DEBUG_OUT(("Data Out Complete\n")); // 20100201 chau //BULK_Read(hBulk, szBulkDataIn, sizeof(szBulkDataIn)); // issue read urb for a next read command if (!hBulk) return; DEBUG_OUT(("E Notify_DataOutComplete\n")); // 20100201 chau } /*static void dma_isr(void* isr_context, alt_u32 id){ IOWR(DMA_BASE, 0, 0); // clear = DONE builtin_wrctl(3, builtin_rdctl(3) & 0xFFFFFFBF); // clear IRQ6 bit }*/ int main() { BULK_NOTIFY Notify; alt_u32 i, data32; // 32 bit input command and decoding pointers pUS_Channel = (char *)&US_Command; 120 pUS_Command pUS_Board pUS_Check = pUS_Channel+1; = pUS_Command+1; = pUS_Board+1; // test pattern: 2K words for receiver memories for (i=0; i