BTeV RICH Data Transfer

Glossary
Event Data Format and Data Transfer
MAPMT Files
MAPMT Files
PMT Files
HPD Files
Example on Calculation of Data Lines

Glossary

It is necessary to straight out couple of uncommon names here:

Photo-detector: Generic name for HPD, MAPMT and PMT.
MAPMT: Multi-Anode Photon Multiplier Tube produced by Hamamatsu. This is the baseline photo-detector for the gaseous system. Each MAPMT has 16-channel arranged in 4×4 array. In total there are 9016 MAPMTs in two 49 (columns)×92 (rows) arrays.
HPD: Hybrid Photon Detector produced by DEP. This is the backup plan of photo-detector to the gaseous system. It has 163-channel readout. There are 944 HPDs arranged in two arrays.
PMT: Photon Multiplier Tube. There are in total about 5000 3″ PMTs arranged in four arrays on four sides of the RICH vessel. We also consider 2″ option.
FEH: Front End Hybrid board designed by IDEAS and Syracuse. It converts analag signal from photo-detector to digital signal. Data sparsification and channel addressing can be also performed on this board by FPGA. One FEH serves up to 8 MAPMTs (8×16 channels), or one HPD (163 channels), or up to 64 PMTs (64×1 channels).
FEM: Front End Multiplexer board that will be designed by FNAL. One FEM serves upto 6 HPD FEHs (978 channels), or up to 4 MAPMT FEHs (512 channels), or up to 4 PMT FEHs (256 channels).
Data Cable: Category 6 (or equivalent) cable. It has 4 twisted pair differentially driven serial lines. Two of them are used to transfer data (data lines).
Data Line: The twisted pair serial line to transfer data. It can transfer event data at 75.7 MB/s, or 5 16-bit words per bunch crossing (132 ns).
DCB: Data Combiner Board that will be disigned by FNAL. It collects data from FEMs and sites at crates away from the detector. Proper event sorting might also be performed.
Channel ID: A number that can uniquely identify each channel of photo-detector. BTeV RICH system uses two parts of channel ID: one that uniquely identifies channel at each FEM and one that identifies FEM.

Introduction

The BTeV experiment has a dual-radiator (gaseous C4F10, and liquid C5F12) RICH detector. The photon radiated by the gaseous radiator are focused onto and detected by MAPMT arrays, or HPD arrays as a backup design, mounted at upstream surface of the RICH vessel. The photon radiated by the liquid radiator are detected by PMT arrays mounted on 4 sides of the vessel.

The photo-detectors: MAPMT, HPD and PMT are connected to Front-End Hybrid (FEH) board which digitizes the signals. The digital hit information are grouped at Front-End Multiplexer (FEM) board and shipped to Data Combiner Board (DCB) via long cables.

This page provides the physical location of photo-detectors (reference: RICH mechanical design page maintained by Herman), ID definition of photo-detector, FEH and FEM, grouping of photo-detectors to FEHs, grouping of FEHs to FEMs, occupancy information and number of data cables needed for each FEM.

Event Data Format and Data Transfer

For each bunching crossing only a small fraction of photo-detector channels have hits. Although all channels are readout by FEH only the hit information are shipped to DCB (sparsification, or zero suppression). Each hit is tagged with a unique ID for the channel.

The RICH channel ID contains two parts: one that uniquely identifies channel at each FEM which is transfered from FEM to DCB to tag each hit; and one that identifies FEM which is added at DCB. In the configuration that will be described below the maximum channels that one FEM serves is 512 for MAPMT system (978 for HPD option) and 256 for PMT system. Thus a 9-bit (10-bit) address per channel at FEM is enough. In total the RICH system has about 360 FEMs (230 from HPD option). So a length of 18-bit address is sufficient for RICH system.

A event data record, as defined by Walter, contains one 16-bit header word providing bunch crossing information and record count, and many 16-bit word of event data. Two bits of each 16-bit word are used to identify the type of the word thus only 14-bit per event data word can be used to transport hit information. The RICH system ships one hit information per word.

BTeV will use catergory-6 cable to transfer event data between FEM and DCB (ref: BTEV Front End Timing, Control and Data Interface by Walter Stuermer BTeV-2260 and BTeV-3274). It has 4 pairs of twisted pair differential lines. One pair provides reference clock at 7.57 MHz (132 ns period). One pair is used for timing and control and 20 times faster of the reference clock. And two pairs are used to transfer the event data.

The event data bit clock is 100 times that of the reference clock. Each pair could ship sequential bit data at 757 Mb/s. The event data are encoded with 8B10B format to balance the current and to provide parity check. Thus each pair of data lines could transfer data at 757 (Mb/s) ÷ 10 bit/Byte = 75.7 MB/s. That is to say it can ship 5 16-bit words per bunch crossing (132 ns).

Currently we use 600 Mb/s sequential bit data rate instead of 757 Mb/s to estimate the number of data lines. This value comes previous version of design (636 Mb/s) by Walter and an extra safe margin requested by Ed Bersoti.

MAPMT

The RICH detector uses 9016 MAPMTs configured in two 49 (columns) × 92 (rows) arrays. Each MAPMT has 16 channels in 4x4 array. A FEH connects to 4 (columns) × 2 (rows) MAPMTs. One FEM serves 512 channels from 2 (columns) × 2 (rows) FEHs. To balance the data rate, at high occupancy region more FEMs are used. In total there are 1196 FEHs, and 334 FEMs, 500 cables with 870 data lines used, connect to 28 DCBs. In the scenario of 6 interactions per bunch crossing with 396 ns time interval we needs less data cables and DCBs.

List of files with detailed informaiton.
mapmt_iden_q1.eps
mapmt_iden_q2.eps
mapmt_iden_q3.eps
mapmt_iden_q4.eps 		MAPMT location and ID definition for 4 quarters seperately. The ID number starts from the bottom row of inner most column, from bottom row to top, then next column.
mapmt_ifeh_q1.eps
mapmt_ifeh_q2.eps
mapmt_ifeh_q3.eps
mapmt_ifeh_q4.eps 		Maps of FEH ID definition, in the similar fashion as of MAPMT ID. Note that the colored pattern indicates different FEHs.
mapmt_ifem_q1.eps
mapmt_ifem_q2.eps
mapmt_ifem_q3.eps
mapmt_ifem_q4.eps 		Maps of FEM ID definition, similar to that of MAPMT ID and MAPMT FEH ID. Exceptions are ID=33-37, where each 2×2 FEH array is split in to two parts and connects to two FEMs (also true for -X side).
mapmt_group.txt This table contains the physical location of MAPMTs. Eight MAPMTs (4×2) connect to one FEH. Primarily four FEHs (2×2) connect to one FEM, later more FEM are added in high occupancy region to optimize load. The columns are as follows:
  1. MAPMT ID (1 - 9016).
  2. Row ID (1 - 92), started from bottom.
  3. Column ID (1 - 49), started from inner most column.
  4. FEH ID (1 - 1196), counted in similar way as of MAPMT ID.
  5. FEM ID (1 - 334), which may be overrid by next file.
  6. X location of the center of the MAPMT window.
  7. Y location of the center of the MAPMT window.
  8. Z location of the center of the MAPMT window.
mapmt_regroup.txt This table contains the final FEM assignment to each FEH. The columns are as follows:
  1. FEH (1 - 1196).
  2. Initial FEM ID.
  3. Final FEM ID. Only in high occupancy region there are extra FEMs added.
  4. Average number of hits per bunch crossing for 2 interactions per bunch crossing. The hits including electronic noise.
  5. Samilar as column 4, except that there are 20% overhead of physics hits.
  6. Number of data lines needed for this FEH, estimated using number of hits in column 5, 132 ns bunch time interval, 600 MBit/s data transfer speed, and 16-bit per hit. No event header is included here as that is for a whole FEM. This is used as a reference to manually optimize the FEM assignment.
mapmt_2int_cable.txt Number of data cables needed for each FEM. Note that not all data lines in the cable are needed. In estimate the number of data lines we assume that an extra 16-bit word is added to each bunching crossing event block. The columns are as follows:
  1. FEM ID (1-200).
  2. Number of data cables (1-4).
  3. Number of data lines (1-8).
  4. Number of data lines that is not rounded. This is just to show how much room left.

PMT

There are 4948 PMTs arranged in 4 arrays on 4 sides of the RICH vessel. They are grouped to 98 FEHs, each FEH connects to an 8×8 PMT array. 24 FEMs are used each connects to 4 FEHs. In total 44 cables are needed with 78 data lines used and connected to 3 DCBs.

List of files with detailed information:
pmt_iden_xp.eps
pmt_iden_xn.eps
pmt_iden_yp.eps
pmt_iden_yn.eps 		PMT location and ID definition for each array. The colored pattern shows the grouping to FEH.
pmt_ifeh_xp.eps
pmt_ifeh_xn.eps
pmt_ifeh_yp.eps
pmt_ifeh_yn.eps 		Maps of FEH ID definition, also showing in colored pattern.
pmt_ifem_xp.eps
pmt_ifem_xn.eps
pmt_ifem_yp.eps
pmt_ifem_yn.eps 		Maps of FEM ID definition.
pmt_group.txt This table contains the physical location of PMTs. 64 PMTs (8×8) connect to one FEH. Four FEHs connect to one FEM. The columns are as follows:
  1. PMT ID (1 - 4948).
  2. Quarter (1: X+, 2: Y+ (top), 3: X-, 4: Y- (bottom).
  3. Row ID.
  4. Column ID.
  5. FEH ID (1 - 98).
  6. FEM ID (1 - 24)
  7. X location of the center of the MAPMT window.
  8. Y location of the center of the MAPMT window.
  9. Z location of the center of the MAPMT window.
pmt_regroup.txt This table contains the final FEM assignment to each FEH, which is the same as in file above. The columns are as follows:
  1. FEH (1 - 1196).
  2. Initial FEM ID.
  3. Final FEM ID. Same as inital FEM ID.
  4. Average number of hits per bunch crossing for 2 interactions per bunch crossing. The hits including electronic noise.
  5. Samilar as column 4, except that there are 20% overhead of physics hits.
  6. Number of data lines needed for this FEH, estimated using number of hits in column 5, 132 ns bunch time interval, 600 MBit/s data transfer speed, and 16-bit per hit. No event header is included here as that is for a whole FEM. This is used as a reference to manually optimize the FEM assignment.
pmt_2int_cable.txt Number of data cables needed for each FEM. Note that not all data lines in the cable are needed. In estimate the number of data lines we assume that an extra 16-bit word is added to each bunching crossing event block. The columns are as follows:
  1. FEM ID (1-24).
  2. Number of data cables (1-4).
  3. Number of data lines (1-8).
  4. Number of data lines that is not rounded. This is just to show how much room left.

HPD

Each HPD has 163 channels, and connects to one FEH. The HPDs are packed in 6 HPD hexagonal modules. Generally each 6-HPD module are served by one FEM. But for high occupancy region, we use 2 or even 3 FEMs per hexagonal module to limit the number of data lines be no more than 8 per FEM.

In total we have 944 HPDs, 944 FEHs, and 200 FEMs. we need 468 data cables to transfer data from FEMs to DCBs. Out of 2×468 data lines only 849 will be used, and 27 DCBs will be need to transfer data.

List of files with detailed information.
hpd_iden_xp.eps
hpd_iden_xn.eps 		HPD location and ID definition for +X array (xp) and -X array (xn) separately. The ID number starts from the bottom row of inner most column of +X array, and then next row till the top, and then next columns. The -X side is mirror symmetric to +X side.
As each FEH serves one and only one HPD, the FEH ID is the same. Note that the colored pattern indicates the hexagonal modules.
hpd_ifem_xp.eps
hpd_ifem_xn.eps 		Maps of FEM ID. The order of FEM ID is similar to that of HPD ID. But for hexagonal models that needs more FEM, ID of extra FEMs are counted later. For example the module of FEM 23 are served by FEM 23, 96 and 97.
hpd_group.txt This table contains the physical location of HPDs. The HPDs of one hexagonal module are primarily grouped to one FEM The FEM IDs at high occupancy region may be overrid by next file to optimize FEM load. The columns are as follows:
  1. HPD ID (1 - 944). The FEH is the same.
  2. Row ID (1 - 30), started from bottom.
  3. Column ID (1 - 15), started from inner most column.
  4. FEM ID (1 - 200), which may be overrid by next file.
  5. X location of the center of the HPD window.
  6. Y location of the center of the HPD window.
  7. Z location of the center of the HPD window.
hpd_regroup.txt This table contains the final FEM assignment to each HPD/FEH. The columns are as follows:
  1. HPD ID (1 - 944).
  2. Initial FEM ID.
  3. Final FEM ID. Only in high occupancy region there are extra FEMs added.
  4. Average number of hits per bunch crossing for 2 interactions per bunch crossing. The hits including electronic noise.
  5. Samilar as column 4, except that there are 20% overhead of physics hits.
  6. Number of data lines needed for this HPD, estimated using number of hits in column 5, 132 ns bunch time interval, 600 MBit/s data transfer speed, and 16-bit per hit. No event header is included here as that is for a whole FEM. This is used as a reference to manually optimize the FEM assignment.
hpd_2int_cable.txt Number of data cables needed for each FEM. Note that not all data lines in the cable are needed. In estimate the number of data lines we assume that an extra 16-bit word is added to each bunching crossing event block. The columns are as follows:
  1. FEM ID (1-200).
  2. Number of data cables (1-4).
  3. Number of data lines (1-8).
  4. Number of data lines that is not rounded. This is just to show how much room left.
hpd_2int_cable.xls Number of data cables needed for each FEM.
  1. FEM ID (1-200).
  2. Number of hits per bunch crossing.
  3. Data rate without any event header. (=column(2)*16/132)
  4. Data rate with 1 word event header. (=(column(2)+1)*16/132)
  5. Number of cables.

Example on Calculating Data Lines

FEM ID 5 It connects to HPDs 8, 9, 38, 39, 67 and 68.
Number of hits per bunching crossing 19.26 9.58 physics hits + 9.68 noise hits
Number of hits with 20%
overhead of physics hit 		21.18 = 9.58 × 1.2 + 9.68
Number of words
to be transfered 		22.18 = 21.18 hit words + 1 header word
Number of bits to be transfered 354.9 = 22.18 × 16bits/word
Data transfer speed required 2.688 GBit/s = 354.9 / 132ns
Maximum speed per data line 480 Mbit/s = 600 × 0.8, as requested by Ed.
Number of data lines needed 5.6 = 2.688 / 0.48
Number of data cables needed 3 = 5.6 / 2 and round up

Please send your comments to: Jianchun Wang