Dana Arenius and Venkatarao Ganni

Westfall:

Maybe we’ll just start with Rao, and tell a little bit… you were educated partly in India?

Ganni:

Yes, I have a Master’s degree in Mechanical Engineering in India — then I came to United States in ‘74.

Westfall:

Which institute?

Ganni:

I came to Madison, Wisconsin in ‘74. I started working at CTI-Helix Process Systems in Westborough, Massachusetts in the helium refrigeration business from December of 1980 after my graduation.

Westfall:

Did you have a bachelor’s degree in India?

Ganni:

Yes.

Westfall:

I see. And what was the University in India?

Ganni:

I have the bachelor’s from University of Mysore, and Master’s from Indian Institute of Technology, Madras. Then my second Master’s at University of Wisconsin, Madison, and my Ph.D. at Oklahoma State, Stillwater, all in ME in the thermal sciences area.

Westfall:

Okay. And then how did you get to JLab? Or did…

Ganni:

After I worked in the industry for 10 years, I was invited to join the Super Conducting Supercollider (SSC) lab in Dallas, Texas and I was asked to head the cryogenic engineering effort there. Because of my industrial and educational background, they felt I would be a good fit to head that group at the supercollider. So I started in December ‘89 until its demise in ‘93. And when the supercollider ended that’s when I got a call from the Continuous Electron Accelerator Facility, CEBAF (Jefferson Laboratory’s former name), because they were in the middle of the 2K process endeavor, which was a major undertaking where they had whatever difficulties over the last two to three years before that timeframe. As I was told, industry had reached a dead end and the lab reached its dead end as well with them. The lab disparately needed the 2K helium refrigerator working, so that’s when they approached me to see whether I would be interested to join, and I said, “Yes, now that SSC is closed, I’d be interested to pursue new technologies.” In January of ‘94 I came here.

Westfall:

Okay. How about you Dana?

Arenius:

I graduated in 1972 with an undergraduate degree, a BS EE in computer design, and at that time I finished school it was very difficult to find work due to the fact that there was a downturn in the economy. I found myself working for an electrical consulting and constructing firm in Massachusetts. With that, I was responsible for design, electrical distribution for chemical plants as well as commercial buildings like banks and office buildings. In 1975 I saw an opportunity to become involved with instrumentation control in a cryogenic company in Waltham, Massachusetts called Cryogenic Technologies Incorporated, or CTI. So my first venture into cryogenics was in 1975. In joining them, I wanted to get a better background in thermodynamics in particular with cryogenics so I took graduate courses at MIT to fulfill that need. Being an instrumentation control engineer with that company I was involved in much of their standard products, which were standard helium refrigeration plants that served small laboratories for experimentation at 4 Kelvin. There was, however, a growing need in larger laboratories for larger machines, and my work found me designing control systems for larger refrigerators, which were provided to national laboratories such as Brookhaven, Fermi, Lawrence Berkley, Oakridge, and Los Alamos. So for the next 13 years I was involved in most of the large helium refrigerator designs, providing to those laboratories. In December 1988, I joined Jefferson Lab because they were planning to use a 2 Kelvin refrigerator, which I was very interested in. It was the forefront of the type of work that I really wanted to be involved in with cryogenics. What attracted me to that was that we had previously to that work with Brookhaven and Oakridge at 3 Kelvin, which is a sub-atmospheric operation, but nothing that was as low pressure as 2 Kelvin. To my reasoning, this was the future of what the laboratories would need. So I understood in joining the effort, I was making a quantum jump, even at this temperature, from previous work, which was about one-fifteenth of the size of what was being proposed for CEBAF. Our machine, for our first Central Helium Liquefier (CHL), was on the order of 235 grams a second, and yet machines prior to that were no larger than 15 grams a second. So it was a quantum step in size and complexity of the machinery, and I wanted to be a part of that. I continued as being the head of instrumentation and control in the cryogenics group until 1999, and then I became the group leader in charge of operations and new developments since 1999.

Westfall:

Thank you for that. In re-reading this document that Claus wrote about the beginning of the CHL… In fact he even has it time-lined here that we can refer to the talks about from the beginning of when it was developed in the conceptual design report for the lab that was then called CEBAF. I was wondering if you could tell me when you both — Now what year was it that you came in?

Ganni:

January of ‘94.

Westfall:

And Dana, when did you come?

Arenius:

December of 1988.

Westfall:

Okay, so you are up near the beginning, so you were actually here in the year that the contract was awarded.

Arenius:

That is correct.

Westfall:

Claus Rode tells the story of the difficulties with the contractors, but maybe you could explain from your point of view. Did you already realize that there was a problem when you came?

Arenius:

I knew there was a great challenge, a technical challenge. The challenge was that prior to this the largest capacity machine at 2 Kelvin was at Tor Supra and was 15 grams a second, and yet this required going up to 235 grams a second. The other challenge other than size is that they use a slightly different technology for Tor Supra where the cold compression did only a part of the work; the rest was done by warm compressors, which is a technology that we had used on other projects prior to CEBAF. This new technology required everything was being pumped by multi-stages of cold compressors all the way up to one atmosphere. That created a new technological challenge which delayed the commissioning of the equipment because industry had great difficulty in being able to characterize what was needed to achieve the full compression ratio required.

Westfall:

I got the impression from what Claus wrote that he thought that part of the problem was that there was this two-part contract, with two different contractors?

Arenius:

Well the two-part contract was one that the main contractor was CVI (Consolidate Vacuum Incorporated). They had subcontracted out the cold compressor design and fabrication to L’Air Liquide. Of course, whenever you have a situation like that, things can run more smoothly or they can run a little more difficult. But I think that perhaps that the companies were not prepared for the technological challenges. This was typically the first type of plant of its kind and size, with this particular technology.

Westfall:

I can tell you that one of the things that was a challenge for the accelerator as a whole was that they were also on the SRF side trying to make the cryo-modules. So the organization was trying to have these two big challenges that had to be done to make this machine work. I was actually there in this ‘93-‘94 period, and I saw a lot of people literally running up and down the halls very upset, worried about the CHL problems, and worried about when the technological problems would be solved.

Arenius:

I think that we addressed those problems because in many cases we relied upon the technologies that we had worked with, certainly not at this low-pressure level of 2 Kelvin, but at 3 Kelvin with both Oakridge and also with Brookhaven machines with cold compressors. As we were going through the commissioning portion of the plan, we saw where the performance of the machine itself was not in accordance with what was expected based on the design. There was reluctance on the part of the vendors to look at that difference and say, “We have to try something new.” That is to say… not to try the same thing over and over again, assuming that their calculations were correct and that’s the operating points. In truth, in engineering that’s your design goal, but sometimes the machine doesn’t turn out exactly like that, and that can be simply result from the tolerances and performances of all the components — when you stack them up you end up with a slightly different operating point. But the vendors in this case didn’t want to explore really where the machine design ended up being in practice, and they kept trying to operate it based upon their calculations, and were unsuccessful for basically two years of trying.

Westfall:

Can you give me a nice summary of what they did wrong?

Arenius:

I think that they didn’t realize the tolerances of the buildup of the components which made up cold compressors and the performance variations from the design basis. That is to say, if you have heat exchanges or you have turbines or cold compressors, they typically have a tolerance associated with their performance, and you can fall within that tolerance. When you sum up all the equipment right, then you can end up with a slightly different operating point then you had calculated. Some of the theoretically predicted to actual behavior of the equipment in this new technology area is also a contributor.

Westfall:

So it’s a matter that the concatenated system has different tolerances than what you might estimate if you took the individual pieces separately?

Arenius:

That’s correct. And we see that typically in industry where you could even have standard product lines, and you have one product come out but it has a slightly different performance than the exactly made item on the assembly line just before it. Some work, a little bit different than others.

Westfall:

And everything had to match in order for…

Arenius:

And everything had to match. But the vendors didn’t know what to do to characterize where the new machine was really can run, and how to adjust basically the controls and the performance, to demonstrate successful operations of the machine.

Westfall:

Go ahead Ganni.

Ganni:

I also worked with Dana on a lot of those projects. We worked for CTI/ Helix, Koch industries together, and we proposed for the CHL project from KOCH industries, and Dana was also a part of it when we proposed it, but later on he came here, but I was still at the Koch industries.

Westfall:

Okay, so you were in the industry side, right?

Ganni:

Yes. And I supplied the Central Test Facility (CTF) helium refrigerator for the cryo module production in ‘89 to CEBAF. During this period Dana happened to join CEBAF, and I was still at Koch when we made our proposal for CHL, and we were not successful. Although our company had the cold compressor experience and low bid, our proposal was not selected on some kind of an evaluation basis. I never understood the evaluation process or the basis!

Westfall:

Not competitive. So at what stage are you?

Ganni:

That was in ‘87 for the CHL vendor selection process

Westfall:

So it was before the contract in January of ‘88.

Ganni:

That’s right. I felt at that time the process was not fair, also when you lose you always feel you didn’t get a fair deal, I guess. A few years later I was invited to come to work at the SSC, and then after it was closed, Dana at the request of Claus approached me to see whether I would be interested to come here. When I started looking at the system as built by CVI/L’Airliquide, I noticed that there were a few critical pieces missing in the installed machine, which were included the Koch proposal for stability between the 4K and 2K systems. I knew because they sent me the flow diagrams for the installed machine for my initial visit to see how I could participate in this program (after the SSC shut down at the end of ‘93). So that was the first thing I suggested, that there had to be some additional hardware to make this system stable.

Westfall:

Okay, what were the additions?

Ganni:

We modified the hardware. We ended up building what we call heat-exchange 9A, which is built and installed to stabilize the process between 4 K and the 2 K systems. And then, because of my thermodynamic background, I asked, “What is the proposed pump down path?” No one in the group understood at that time. Then we also recognized that each cold compressor speed cannot be controlled and managed by its own individual temperature and pressure measurements. There are fourteen measurements used for controlling the four cold compressor individual speeds. My instincts from the other plant operations told me that this is a problem. I recommended to the group that we cannot control cold compressor speeds using fourteen measurements, because each one has a variation, and they cannot exactly represent the real process what you expect at every instant. You can have a theoretical idea, but in practice what you get will be different. So I said, “We’ve got to reduce the number of variables to a very few,” and eventually we did. By the time they accepted those ideas, we had been in the CHL control room for probably a month or so working around the clock from the beginning of April ‘94. I remember we just lived in the CHL control room until the system is finally commissioned in mid of May ‘94.

Westfall:

Okay, to make sure that I understand how does the sequence go.

Ganni:

As I was told, CVI supply the equipment in 1990 to CEBAF. The equipment is installed here in ‘90 and ‘91. Then CVI/ AirLiquide tried to start the equipment. They got the 4K systems going in 1991, but they were not successful to get the 2 K system operational from that time on. The 4 K was supporting the pre-commissioning activity of the cryo-modules (CM). The CM commissioning was limping along using the vacuum pumps. So by realizing that the cold compressor system may never make it and they started a full pledged warm vacuum pumping system called K100 for the 2K operation of the CEBAF with a large resources commitment.

Westfall:

Did you have something to add?

Arenius:

Yes. In here you get into the difficulties of the contracts and technical needs. One is under a contract you have to provide vendors opportunity to demonstrate that a machine works, and so that you can voice your opinion about items. Early on, when we were looking at the pump-down procedure with the cold compressors that we were proposing, I raised the question that it appeared to me that they were trying to pump down through a surge location in one of the stages of the cold compressors, and that would cause a failure and they would never be able to get beyond this point. Yet the vendor looked at my calculations and said, “Well, you’re right, Dana, but we think we’re going to be okay. And you have to give us the opportunity to demonstrate that we’re going to be okay.” So you get into this situation where you know something different that is —

Westfall:

But it has to be documented…

Arenius:

But it has to be documented, and you must give an opportunity to the vendor legally to actually demonstrate that —

Westfall:

In the meantime, you’re trying to make this machine work.

Ganni:

The difficulty is when you say it’s a turnkey, that once you write the contract you have very little control. You can’t influence them, no matter what you think, unless it is part of the contract. It is very difficult to predict and put all these links in to the contract and especially for new technology. That is why Ray More from our company Koch asked CEBAF management to divide the cryo system into two parts. One standard 4K part and the other a joint R&D project between the industry and the Lab. As vendors continue the commissioning effort with various ideas, the time was running out for the project. Then in ‘93 CEBAF cryo group finally recognized that almost three years had gone by and the vendor hasn’t figured it out. They’ve made attempts to solve the problems, but were not successful. I heard Dana recommended strongly to close the contract. At the same time frame SSC was shut down. They approached me to see whether I would be interested in participating in this adventure. I was approached by many organizations, but I chose JLAB because of the technology interest. I’m aware that in part they wanted me because I had been part of the Koch bid as the process analyst for this procurement. And I was interested to see how we could succeed in implementing the new technology, which I was curious when we were making the bid.

Westfall:

Okay, I would like one more little good quote about why the technology was so attractive.

Ganni:

Helium cryogenics itself is technically interesting. The field started developing after the Second World War in the U.S. at MIT. The practical helium liquefier’s production started at CTI Technologies, where Dana and I worked. That led to the helium cryogenics in a larger scale in the U.S. Sam Collins and Bob Johnson supplied the first 300 watt 1.8K machine to Stanford LINAC in the late ‘60s. That was the first 2 Kelvin machine of that class.

Westfall:

Was that at HEPL?

Ganni:

I am not sure, but may be. That is where I met Mike McAshan, who invited me to SSC and later I was invited to JLab.

Westfall:

Okay. Let’s go back to this crunch time in I guess the early ‘90s — part of the refrigeration system works, but not the whole thing?

Arenius:

It did work, as I recall. But we basically had to pump down to 31 thousandths of an atmospheric pressure to achieve the performance requirement, and they could pump down to about 70 thousandths of an atmosphere. But then they basically ran into a stone wall, and they’d trip the cold compressor — they could not pump down any lower than that.

Westfall:

And you needed to get down to 2 Kelvin in order to get the cryo units to operate so that the accelerator could operate.

Ganni:

Close to 2 K, but below lambda point.

Westfall:

They got part of the way to the goal, but and because of contractual reasons, since it was a turnkey, you had to let them try to solve the problem on their own. But when they didn’t, you had to find your own solution.

Arenius:

Yes. We had in the back of our minds that we needed to go in a different direction. But I think that the turning point here is that basically as a laboratory we were running out of time.

Westfall:

So that’s when you came, and you came interested in this technology, and also actually having had bid in from Koch on it before. So you said, “There are these pieces missing.”

Ganni:

I didn’t know what was missing because I didn’t know what they bought. See, I had never seen the final design what CEBAF ended up buying.

Westfall:

But once you got here…

Ganni:

Once I could see the devices, I knew some of the necessary pieces were not there. That’s the first thing I said, these are some things we must do first, which they did right away. Lab started the parallel process to install a warm vacuum system, and the other to get the cold compressor operating. Now, because cryo staff at CEBF were trying and living with it for two to three years before I got here, they had a lot more observed knowledge than I had. So there were some difficulties, to see where the trips are happening from my theoretical aspects. And so Claus, myself, Bill Chronis — we all had a lot of arguments, in that month and a half as they have their observed points, but from my theoretical aspect I could see some, but I couldn’t see all of them in that short time.

Westfall:

It was very intense, wasn’t it?

Ganni:

Very intense because we were living 24/7 at CHL. My family was still in Dallas; I was here. Whether I would go to the hotel or stay at CHL didn’t make any difference for me. So I was practically living in the CHL control room. Sometimes I said that we had to let the machine go to where it would naturally go with the equipment we had rather rely on what was initially predicted by theory, because some of the equipment do not represent what the theory assumed in the design basis. And that statement was very difficult for most of the people here to absorb, saying, “How can you say a machine can work better away from the design TS point?” The controversy and the arguments were very heated. The floating pressure was part of it because I already implemented the basic technology at SSC; which naturally allows the actually installed components of the machine to go to its maximum capacity in an optimal way. The folks at CEBAF were unaware of it and had hard time to accept it as a possibility and that was my basic point.

Westfall:

So the thing that you’re famous for is this floating pressure Ganni cycle?

Ganni:

That’s right; it was already used to some extent at SSC. Well, it was not that mature at the time.

Westfall:

But that’s when you got on —

Ganni:

I knew the concepts of the floating pressure I was working with at SSC in ‘91-‘92, and I knew some parts of that were required in the CEBAF machine to make it work, which were not proposed by any vendors, or recognized by the people that were working in this field at that time. They were saying that ideally you come up with a plan for how to make the device work. But as you choose the pieces, as they are put together, that original plan may not be the right way to make the device work to meet your needs. Because of the complexity, it’s very difficult for people to even think that there is a better plan for the equipment once it’s put together. Due to too many complexities with these systems, nobody considered these aspects before my involvement with this floating pressure technology. When I went to the SSC as a user instead of as a vender, I recognized that once you build a system, although you have the plan, you have to put that aside and start seeing what will work with the actually built hardware and the actual test needs; which can be substantially different from the original plans. That was the recognition and the basis for the floating pressure process development.

Arenius:

To add something more to this, having come from industry, we understand the weaknesses and the strengths of the industry. We also understand basically the technology of what we know and what we don’t know. I think there’s a tendency for end users, the laboratories, buying equipment, to think that industry knows the ultimate solution and it’s always 100% correct. Being from industry, we know that that’s not necessarily true.

Westfall:

Sometimes there’s more R in the R&D than… [laughs]

Arenius:

So there was a feeling that industry understood completely all the technology that was necessary to make a successful machine here. What was difficult to bring across to management here was since we came from industry we understood that this work was a challenge to the industry.

Westfall:

Well I can tell you from knowing the other aspects of the machine; they would certainly have liked to be able black-box something, [laughs], because they had whole other issues from funding, to the SRF.

Arenius:

There was a great focus for the challenges of the SRF portion of the machine, and I think at the time, for whatever reasons, there seemed to be the idea that gee, maybe the technology at the cryogenics also needed to be cutting edge.

Ganni:

The 2K was not recognized as one of the challenging technologies for CEBAF although Ray More from Koch stated that. The folks at CEBAF underestimated how difficult it would be to make this technology work. It is clear; from the effort went in that was not recognized as a new technology. To me, going back, I see that the first time that I went from being a vendor to being a user changed my world. I started seeing how much variance in utilizing the refrigeration system compared to what the users had specified in procuring them. Knowing the equipment potential as a vendor and experimental needs as a user was the starting point for the floating pressure technology idea. At SSC, the initial planning at Berkeley was based on three independent machines, a liquefier, a refrigerator and a shield refrigerator; at each of the ten locations. When I proposed a more efficient, less expensive, reliable integrated machine, some in the industry at that time were saying — that it’s selling snake oil. How can it be the efficient system is the least cost and the most reliable? I was taken to task for making that statement. But I was proved to be correct. So we built the magnet test machines at SSC, which were one of the first for wider applications based on floating pressure theory with automatic interchangeable equal Carnot Capacity refrigeration and liquefaction capacity. When I came here I saw that was one of the things required here was to move away from the design TS diagram on the 4 K system to provide the capacities and the turn-downs and to handle the interactions between the 4 K and 2 K systems. That was not part of the original plan and met with resistance. And I said, “Naturally this equipment will adjust itself if we allow it.” That realization was also not there on the user side or supplier side at that time in the early ‘90s. That’s where I had a hard time convincing industry and the users, saying that once you build the system — Yes you had a good plan (the design TS diagram), but put that aside, don’t use it as your driver but only use as a guide line. Try to see what you actually got built in hardware and your actual experimental needs and see, from there move on, rather than use the original TS plan and try to force the equipment towards it, against its natural will. My thinking is you have to develop a new plan with the equipment you got, so you have to correct the original plan based on the actual equipment, to move towards your goal. That was the whole floating pressure idea.

Arenius:

One of the things I think that was missed from a technological standpoint is that a 2 K helium refrigeration system requires an extreme turn down of the 4 K refrigerator. As we make a transition from 4 K liquid to 2 K liquid, we’re actually recovering cold gas. That is, we’re going sub-atmospheric, but we’re recovering cold gas.

Ganni:

We have excess refrigeration.

Arenius:

We have excess refrigeration — we don’t need the four and a half Kelvin refrigerator at all. And the question is how you balance it thermodynamically where you’re transitioning from not needing any refrigeration to needing all your refrigeration capacity as you are transitioning from 4 K to 2 K. That capacity variability of the refrigerators is in deep contrast to the typical refrigerator that industry would provide, it would be basically for liquefaction, having one operating point, and it’s operating point is usually at maximum capacity to fill tractor-trailers full of liquid helium for sale. That’s where most of the business is in the helium business. Yet the machines required by the laboratories have a requirement to be much more versatile than that, covering a wide spectrum of refrigeration and liquefaction loads and capacity in various combinations. Until coming to CEBAF and actually being on the user side, I did not realize how little engineers in industry understood that need.

Ganni:

See, the priority in the industry is to produce the refrigerator or liquefier as requested by the user, collect the check and moved to the next project. We rarely had the opportunity to look back and see how the machines we sold are actually used, because we were always trying to sell the next machine. The labs that needed these multifunctional cryo systems treated them as a utility and less interested in funding the developments in the support technologies.

Westfall:

Okay, you somehow solved the problem in…

Ganni:

The beginning of ‘94. We have the equipment built by the industry. But now it was up to us at the lab to make it work. So we discussed and tried many things. Claus, Dana, Bill, all these people used to say, “Okay this had been tried.” My proposal was to allow the system to move away from the original 4 K TS diagrams since some of the major components like the main warm compressors have been replaced with larger compressors. So we went through these discussions to see how to get the most capacity out of the installed equipment on hand when needed. And similarly we cannot use all of the instrumentation on the cold compressor system because the variations in them will never let you close a loop.

Westfall:

I thought you said at one point that there were two separate lines of development.

Ganni:

That was correct. A completely different system based on vacuum pumps is being pursued in parallel. That’s not part of this cold compressor technology endower. That was tried as a completely parallel path in case we never make this cold compressor system work.

Westfall:

But you did. [Yes.] Okay, so that I understand in detail, what did the vendors leave you with, and how did you proceed from there?

Arenius:

They basically left us with their hardware and basically their calculations of what they were trying to do, which only allowed for partial pump down to where we needed to be. In other words, they left us with their control philosophy and their hardware, and basically nothing else.

Westfall:

Okay, so then you had to make it work.

Ganni:

We had to do that, and we already added the missing pieces. We had to change the direction they proposed. We did not use their pump down scheme because that was not successful; as they tried for a long time. So we had to basically find our own ideas, and collectively we all put together each one, and everybody was given a shot at what they thought was correct. Of course, like I said, we had a lot of heated discussions. But at some point, I still remember when I said to change the main compressor medium pressure from the design 3.2 atm to 2.5 atm, at 11:00 pm, Claus finally gave up and said, “Yeah, go for it,” I could see his frustration and nonbelief that it can result in any improved performance. Naturally, it is very difficult for anyone to act on such a statement. But like I said, it was midnight and we are all tired. So he said, “Go for it. Why not try?” We reduced the medium pressure, and we made more progress. And in that way we made inch-by-inch progress with the changes we introduced.

Westfall:

Exciting.

Ganni:

Well, yeah. That month and a half was probably one of the most exciting periods for me, trying to make this work, from the beginning of April to the mid May of 1994 when we made the cold compressors work. I think it was May 15th.

Arenius:

Yes. There were challenges along the way. Even though one of our biggest milestones was of course to be able to get to 0.031 atmosphere at 2 Kelvin in a pump down, we didn’t really have the reliability that we wanted. We were about 75% reliability with this 2K system.

Ganni:

After successful commissioning, he’s talking about.

Arenius:

So we recognized that there were other issues other than just being able to achieve 2 K, is now we were challenged with making it stable and reliable. That set the stage for the next number of years, after ‘94 developments; we took the technology and moved it further along.

Westfall:

So you have this exciting period. If you could just give me a nice summary, one or both of you, of how you solved the remaining problems with the cryo system.

Ganni:

From my view point, like I stated early on, putting the design basis aside and using the installed equipment to its full potential was one of the recognitions. And adding the missing pieces like HX9A was another key thing.

Westfall:

The missing pieces like the…?

Ganni:

Heat exchanger 9A. Because one of the heat exchangers (HX-9) in the 4K system was designed handle both single and two phase flows, resulted in unstable system. So between 9 and 10 we added a HX called 9A. This was the missing piece that was included in our proposal from Koch, which was not in the CVI/ L’Air Liquide proposal. Another part of the solution was the elimination of many of these instruments from control loops that were used for controlling each cold compressor, and minimizing the number of control points for it to characterize. At the end of the day although low price Koch bid was non-competitive on the evaluation basis, it followed Ray More’s advice of R & D partnership between the lab and industry, utilizing the missing pieces proposed in the Koch bid and with the people like Dana and myself from Koch experience to be a part of it to make it work.

Arenius:

With that instrumentation, particularly with the 2 Kelvin cold compressors, we had four stages in series. And our control system was very complicated. It relied upon 20 different sensors with precise measurements to make this work. And although on paper this looks like it should work, if you had a slight drift of a calibration of one of these instruments or one of those instruments was faulty, then the whole process would collapse. So one of our improvements was to develop a much more simple control philosophy relying upon much less instrumentation to improve our reliability, which was at 75%. And then as we basically made these improvements in terms of how we controlled it. So we went in and changed the devices that we knew would be much more reliable. Thanks to minimal reliable instrumentation like pressure and temperature measurements for controls, as well as a new control philosophy, over the next few years we built that reliability from 75% up into the 90%. The 24/7/365 operations we’ve supported for many years here, basically demonstrating very successfully the reliability of 2 K machines of this technology, not only for CEBAF, but for applications in other laboratories as well.

Ganni:

My memory is that after ‘94, yes, we operated the accelerator at 2K, but as Dana said, the reliability of the helium refrigeration was near 75% for some years. Most of the time if cryo is down, the others were unable to trouble shoot their systems either, and that was not acceptable for the CEBAF operations. So what the management decided is that we have a lot of spare parts bought for this cold compressor system, and some, not all of them, quite a few were existing as spare parts. So management said to the cryogenics group, “Okay, why don’t you guys go ahead and build another parallel 2K system?” The cryogenics group took the challenge, and we bought whatever was missing, and we built the system with many other improvements. That’s the system we commissioned I think at the end ’98 or ’99 timeframe. The new 2K system basically improved the efficiency by more than 10%, although we used the same main components. Each 2 Kelvin watt was probably $4,000 as the first cost basis at that time. By redesigning we improved the 2 K capacity by more than 500 watts or an equivalent of $2 million and the reliability of the machine went past the 98% and presently it is better than 99% and in some years it is 100%. Some of the improvements we provided were the breathing space required between the compressors, added flow stabilizers, increased the heat exchanger size. Even today, the only systems which utilize complete cold compressors to bring the sub atmospheric flow to positive pressure are CEBAF and SNS in the world. Of these one was built by industry and later modified by JLab and the other two, one for JLab and one for SNS were designed and built by JLab. Even today, the JLab 2K system is the largest single machine in the world.

Arenius:

And from a control standpoint, up to this point these large machines had to be manned 24 hours a day in three shifts, and if you were running as long as CEBAF, with close to 365 days a year, you recognize that it is a very large operational staff required to control and operate a refrigeration system. We saw this as another challenge for Jefferson Lab with a new technology. So from the start in ‘94 we started to automate our controls and operations. And so today, where this technology is being used at SNS and Jefferson Lab, for the most part, our control rooms are unmanned, and that’s in deep contrast to other laboratories, both in Europe and in the United States. To large extent, we’re able to operate our refrigerators with about one-third of the staff required at other laboratories for the equivalent number and size of machinery.

Ganni:

JLab cryo system has higher Carnot refrigeration capacity with added 2 K complexity and support 24/7/365 operation and as compared Fermi or Brookhaven cryo systems which have larger cryo staff then we have at JLab.

Westfall:

So that saves money for the organization?

Ganni:

Not only saves money, also it increases the reliability, because computers don’t make changes based on feel where as people do. It is contrary to the belief that it may reduce the reliability. If the machine is properly programmed so that it takes advantage of all of its capabilities, the machine can do better than a person can on a repetitive and consistency basis.

Arenius:

So this brought automation. The process we have now has more than twice the pressure stability of other 2 Kelvin machines in the world, and the reliability is very high. I think that when you look at the contributions of cryogenics in supporting research here in the US, this went a long way in setting the stage for future machines and the success of other projects, which are being developed by the Department of Energy.

Westfall:

Such as?

Arenius:

Such as FRIB at Michigan State University.

Ganni:

JLab upgraded the MSU cyclotron cryo system in ‘98-‘99. I spent a lot of time at MSU in making that system installed and commissioned. The main cold box was built at CTI/ Koch, was used to operate liquid helium production at Bureau of Mines in Amarillo, Texas. We took that machine, refurbished it here at JLab and installed it at MSU, and modified the cryo distribution system in the ‘99 timeframe.

Westfall:

I see. So if you had to again characterize these improvements from ‘94 to ‘99, that consisted of making it more reliable?

Ganni:

And efficient.

Arenius:

Reliability, efficiency, with less manpower. The efficiency improvement helps in the reduction of operating costs. We use liquid nitrogen to pre-cool the helium for the refrigerator, and we see improvements in that area, too.

Ganni:

Well, let’s go back in the timeframe, I guess. May ‘94 we finished the 2 Kelvin, and then we took on the ESR, the end station refrigeration, which supported the three experimental halls.

Westfall:

Okay, end station refrigeration…?

Ganni:

End station refrigeration to support the three experimental halls, A, B, C.

Arenius:

It is ESR, end station refrigerator, it’s a 4 Kelvin machine, and as a 4 Kelvin machine it was capable of 1500 watts at 4 Kelvin, originally built for Lawrence Berkley Laboratories in 1976 to support the ESCAR project.

Ganni:

Dana’s first cryo project at CTI.

Arenius:

Well that project, as I was just a few years out of college at that time, but I was on the design team in the industry that designed the helium refrigeration system for ESCAR.

Westfall:

So this is what you’re doing in —

Ganni:

In ‘95 we took that machine and modified to incorporate 80K purification beds, and that’s the first time at JLab we implemented the full floating pressure technology on that machine as compared to the partial implementation at CHL.

Westfall:

The Ganni floating pressure —

Ganni:

The technology was incorporated in to the ESR machine in ‘95 at an improved level than what we used at SSC.

Westfall:

Is that right, Dana?

Arenius:

We should clarify that there are portions of the Ganni cycle design that can be used for existing machinery that would be ideal at the beginning stages of the design for what we refer to as the full Ganni cycle. However, there are other refrigerators out there, which you can’t take all the advantage of the Ganni cycle because the compressors are already designed, the cold boxes are already designed, the hardware is already there, so they don’t have all the necessary features for the equipment needed to make full use of the Ganni cycle. However, there are significant portions of the Ganni cycle which can be applied which don’t require the change of hardware at all and yet will substantially reduce the utility, the electrical power, cooling water and everything, as a major portion of the full Ganni cycle, and provide better stability and reliability as well. That was the technology that was used at the end station refrigerator that we then incorporated into the CHL and also incorporated at other laboratories such as Brookhaven National Lab to reduce the cost of their operations as well as improved their reliability.

Ganni:

Although some of the concepts involved in the Ganni Cycle were started at SSC, and some of these ideas were applied in the CHL commissioning by allowing the pressures to adjust; were not the full extent of the Floating Pressure Technology as we use it now. The improved understanding of the Ganni Cycle is recognized after the implementation at ESR in ‘95, We started ESR operations from the beginning with floating pressure technology. Then we went back and modified the test lab (CTF) machine to the extent it can utilize it.

Westfall:

CTF is?

Ganni:

Cryogenic Test Facility which supports the SRF group’s VTA and cryo module testing.

Arenius:

Both in development of SRF technology, such as super-conducting cavities, as well as testing a fully assembled cryogenics module performance prior to it going out and being installed in the accelerator.

Westfall:

That’s interesting.

Ganni:

That’s the machine I supplied in ‘89 when I was working at Koch industries.

Arenius:

You know, in industry we understood that there were a couple of key items in the operation of the plants which should be of importance to the laboratory, and that being the cost of the utilities and the reliability as being at the top echelon of those requirements. We understood at the Central Helium Liquefier that the machine that delivered to us by industry was drawing something like six and a half megawatts of electric power, and along with that came a substantial amount of liquid nitrogen usage and manpower, basically manual operation of the refrigerator. What drew our attention was that it goes back some problems which we had in industry back in the ‘70s, and some solutions to some of those problems that we instituted, and then some problems that we did not address and needed to be addressed by industry for future machines. What I’m speaking of is in the ‘70s there was a transition going from warm helium compressors which used reciprocating piston technology to oil flooded screw compressors. All facilities needed to get away from the pistons type of compressors because of extreme reliability problems which caused maintenance and overhaul, which was very costly and did not meet the sort of reliability needed. So what was developed was the application of oil-flooded screw compressors. Now at that time in the ‘70s we were making by today’s standards small refrigerators, although at that time they were considered to be quite large — 300 watt at 4 Kelvin-type machines were considered to be a fairly good size refrigerator. At the time they adopted the oil-flooded screw compressors with pressure ratios across the warm helium compressors, which were compatible with the existing process cycle thermodynamic design that was used with piston compressors. As we tested and developed these in industry, we saw that they were less efficient than the piston compressors for the same thermodynamic process. But these machines were relatively small, and so the penalty you paid for electric power was much smaller as compared to the size of the machines.

Ganni:

And the reliability.

Arenius:

And the reliability. So the industry at that time, of course made that transition to oil-flooded screw compressors, and they also tried to tackle the problem removing the oil that comes out of the compressors. Basically nothing was done to say, “Okay, but these compressors are less efficient than what we had before.” Therefore that affected the overall efficiency of the refrigerators. Really what was needed was a process which matched the operating peak efficiency points of oil-flood screw compressors and the cold box.

Ganni:

See, to put it another way, if you take 100% of the losses happening in the helium system, typically one-third of the losses happen in the cold box and two-thirds happen in the compressor. But people were not paying much attention to the two-thirds. Most of the people who were selling these refrigerators were the cold box manufacturers and turbine suppliers. Their interest has been in the area of the cold box to improve the efficiency. Although the majority of the loss in the power was happening in the compressor, this factor did not catch a lot of people’s interest. JLab designed the helium refrigerator processes for NASA James Webb project and for the JLab 12 GeV project to match the cold box and compressor efficiency characteristics. JLab has put significant effort into modifying and improving the compressor skid design for helium applications with the funding from NASA, and we continue to work on it. We commissioned the NASA helium compressor in May 2012 and the cold box in August 2012. And we will be commissioning our CHL-II shortly. These systems will set new standards to the overall efficiency, operational range and automatic efficient adoption to the actual loads.

Westfall:

One of the things historians do, we like to periodize. So it sounds like we have a natural beginning to 1994 period, which is getting —

Arenius:

Getting basic operation of 2 Kelvin machine, very basic.

Westfall:

So that’s to ‘94?

Arenius:

To ‘94, and at that point we still suffered from availability, reliability; we were using a very large staff to control the cryo system. So other than getting the basic machine to work, we had not at that point in ‘94 to address improvements of efficiency or stability. As we developed our own cold compressor cold box, that was really the beginning of this thought process of trying to pick up where industry in the U.S. had basically left the technology on the table, and that was it. Pick up the problems yet to be addressed that we knew were weaknesses, recognizing that in the U.S. there was no industry to move that technology forward.

Westfall:

So that’s ‘94 to ‘99?

Arenius:

Well it was continuous from ‘94 to today.

Westfall:

Okay, so you built something. I picked up the ‘99 from —

Ganni:

We built the new 2 Kelvin cold box and commissioned that in ‘99. With that we pushed the availability of the cryogenics system to about 98%.

Westfall:

Okay, and so you get something that allows basic operation of the machine. Then from ‘94 to ‘99 you get this machine to work much better — with all the technology that you have to develop to get this cold box in ‘99.

Ganni:

Yes. We built it at JLab, to make availability and reliability and efficiency goals of the 2 Kelvin system.

Westfall:

Okay, but ‘99. Now it’s 2012, so…

Ganni:

See, in ‘94 we commissioned CHL 2 K system, in ‘95 we upgraded and commissioned the end station ESR machine got from Berkeley to support the experimental halls, utilizing the floating pressure technology. In ‘96-‘97 we built the standby refrigerator to support the CHL maintenance. In ‘99 we commissioned the new 2 K cold box built by JLab. In ‘98-‘99 we helped MSU cyclotron cryo system upgrade.

Westfall:

So JLab become a cryogenic resource for the nation after you built the 2 K cold box.

Arenius:

Our technology was now moving outside the laboratory, so at Michigan State we had a requirement to upgrade their cyclotron refrigerator, and we were having difficulty in acquiring a machine for that. They heard about a machine that both Rao and I worked on and industry provided to the Bureau of Minds in Amarillo, Texas, and they wanted to modify that machine for their application. The Department of Energy and through NSF asked Jefferson Lab to basically take on that role of providing a refrigeration system to Michigan State. At that point is where we started to show outside the laboratory the floating pressure technology that was developed here at Jefferson Lab and the reliability improvements for cryo systems. MSU was the first machines, outside of Jefferson Lab, which started to see the remarkable improvement in reliability and operational costs for the cryo systems.

Ganni:

In 2000, SNS was finalized and ORNL was selected for its location. DOE again asked Jefferson Lab to head that. Both the SRF and the cryogenics effort at JLab were basically focused on the SNS project.

Westfall:

I knew about the SRF piece, but I didn’t actually know about the cryogenics piece.

Ganni:

It was a complete turnkey. JLab was the central engineering contractor for the cryogenics system for the SNS cryo system. We basically laid out what the cryogenics system requirements needed to be and we procured some of the components from industry, and we fabricated the 2 K box and some additional subsystems at Jefferson Lab. We provided all the designs for the cryogenics system in the tunnel and outside. We integrated the total cryo system with the cryo-modules. And we commissioned and put it online using the technology we developed at Jefferson Lab for 2 K system and the Ganni Cycle floating pressure technology for the 4 K system to work with all the capacity variations need for the 2 K system during pump down. The cryo system was commissioned in a remarkable time. The 2 K system was commissioned in two weeks, instead of two and a half, three years it took at CEBAF, because of what we learned along the way. During the 2000 to 2005 timeframe, we were fully involved in SNS cryo system in addition to operations at JLab cryogenic systems. By that time in 2005, our 12 GeV kicked off. At the same time in 2004, NASA heard about what we do here, so they have the James Webb telescope program on the horizon, which requires a 20 Kelvin refrigerator to be used for the large chamber, which is 60 feet diameter by 110 foot tall, where a tennis court sized telescope has to be put inside and tested in the 20 K environment. They needed a very stable operating system, with very large capacity and at varying temperatures for cool down and smaller capacity for steady state operations to test this telescope. It’s going to be operated in a million-mile orbit. With the Hubble telescope experience they knew that no shuttle can go to fix their problems because it’s million miles away. The requirements became more stringent when it came to testing aspects. So the industry told them and sent them to Jefferson Lab to get the input if they needed that kind of a stable operating machine. So we were involved on that end. We just finished commissioning NASA machine in August 2012 which has set another performance record and milestone. Actually before that NASA bought in the mid ‘90s, two refrigerators from the industry to do this kind of process from Linde, which were not satisfactory for this kind of stable operation. They wanted better than 0.25 K stability at the load. Their existing machines were giving 2.5 K variation and they were not able to control it any better. So that’s when they asked us to help them fix that, which we did in 2008. We adopted the Ganni Cycle floating pressure technology and it resulted in 45% to 100% of capacity at constant efficiency, and at the temperature stability better than 0.25 K at the load. That’s when they asked us to design the new machine, with a capacity of 13-15 kW at 20 K and 100 kW at 100 K. The problem there is when they started this project, they could not really put their hand on the refrigeration capacity they needed because there were so many variables in the testing needs of the telescope.

Arenius:

This is very analogous to laboratories as they’re developing for instance the new 2 K cryo-module. They do the very best job at trying to figure out what is the load for the refrigeration, and they have an idea but it’s not exact. When they size a matching refrigerator to their estimate, best guesstimate of what their load will be, it may very well be that could be the load and that might not be the load.

Ganni:

Not only that, they add typically another 50% refrigeration capacity margin on the top of it because there are so many uncertainties.

Arenius:

But when industry gets just one number and says to provide 4,000 watts at 2 Kelvin, they design the peak efficiency at that number. Anything other than that number, the efficiency drops off dramatically. So what happens then is —

Ganni:

But cryo system the design capacity also has 50% margin. In practice, all these typically make the system work at off design or with wasted refrigeration.

Arenius:

So you have a situation where you don’t know exactly what the load is going to end up, and you have a situation of a refrigerator that when you stack up all the performance tolerances of all its components, it may not want to operate at that design goal of the refrigerator design. So you have two things, which really say that yes it’s possible that the refrigerator is most likely going to meet your load requirement, but it may not be the most efficient at that load. And so what technology can you use that provides no matter where your load ends up, or where your refrigerator design ends up, you can meet the load at a nice high efficiency? With the Ganni cycle you can say yes. That’s what the Ganni cycle brought to the table.

Ganni:

Basically with the Floating Pressure Technology, we can design these machines with constant efficiency from 100% down to 30% of capacity. Actually in testing it showed that we have ten to one turn down at the high efficiencies. We are pushing the technological goals of maintaining the same peak efficiency, whatever your load is. So it’s becoming less and less relevant for people to be accurate in the load estimate, which has always been a very difficult process. Also in the same period of time, in the early 2000 on, the energy costs and the realization in the community about the importance of saving energy substantially grew. Especially with the 2003 blackout in New York, Brookhaven National Lab was asked to look over their use of electric power for the helium refrigeration system. The RHIC refrigeration system was operating for ten years using 9.4 MW or more input power. So the DOE asked them to get an audit and see if they could do anything about the utility. They came here to Jefferson Lab to review their operations. Initially they were very skeptical that anything could be saved because they were convinced of their operation to be the possible maximum efficiency point like many labs. The industry reviewed their operation with no suggestions for improvements. They didn’t think there was anything left to be optimized. We said, “Not so fast. Let’s have a look at it,” which they reluctantly agreed. And the first time with zero investment we reduced the input power from 9.4 MW to 7.2 MW by changing to floating pressure. Each MW in Long Island is close to three quarters of a million dollars a year. At that time they recognized and asked what else could be done. Dana and I went there and said, “Well, the first two and half megawatts is free, but the next one requires probably some investment.” They asked, “What size of investment?” The answer was that it may be in the order of a pay back in a couple of months of your operational cost savings from the improvements. We incorporated the improvements in phases to match with their operations. The final result was, the operating power for the refrigerator was reduced from 9.4 MW to 4.8 MW. Of course, we made 2.2 MW using the floating pressure and the other improvement came from the increased Carnot steps to bring it down to 4.8 MW. This also resulted in increased beam stability, reliability and reduced operating costs. So that was one of the major contributions that happened between 2004 to 2008 at BNL. In the same period at NASA, which is still continuing, we also introduced other technologies.

Westfall:

And when do you start the 12 GeV?

Ganni:

We started the 12 GeV main compressors in February 2012, and the refrigerator commissioning to follow in April.

Arenius:

But to put that in context, when we first achieved 2 Kelvin operation in ‘94, the CHL was drawing something like six and a half megawatts worth of electric power, regardless of where the load was. With our work at our end station refrigerator, we said, “Okay, we can apply a portion of the Ganni cycle, floating pressure technology” to the CHL, so we can save power.

Ganni:

Applying the floating pressure technology, we dropped CHL electric input power to 4.2 MW from 6.5 MW.

Arenius:

Yes, for 4 GeV operation, and then for 6 GeV it was 5.2 megawatts. So that was a big money saver for utilities here at the laboratory.

Ganni:

We pay more than half a million per megawatt per year in electric bills. We reduced around two megawatts of electric power for the cryogenics systems by adopting this cycle to ESR, CHL, CTF. That is more than a million dollars every year in just in power bill, and wear and tear, environmental impact — they all follow the same ratios.

Arenius:

Since this past decade we’ve been approached by other laboratories, in particular SNS, Brookhaven, MSU, NASA, which is another government agency, basically to provide them cryogenic engineering support for their projects.

Westfall:

So JLab gets involved at what is sometimes called work for others (WFO)?

Arenius:

[Yes.] Very often.

Westfall:

So you get a contract, so JLab is enriched? [Yes.] They loan out your expertise, your experts and your expertise.

Ganni:

Our goal is like this: we recognize we cannot manage and operate and help every lab. Our goal is to bring them here whenever possible. For example, SNS engineers worked with JLab engineers side by side for three years and participated in the design and procurement phases. They went back to SNS to support the installation, commissioning and later in the operations. That is when we handed over the keys for the SNS cryo system, made them more comfortable in operating their system. Same thing we did with BNL also with NASA. Although we have the technology here, we bring these partners here and we train them. Same thing is presently happening with FRIB. We have MSU folks sitting side by side with us at JLab.

Arenius:

Besides working with the end-users, we took the Ganni process cycle floating pressure technology and licensed it to the industry. So it’s a multi-prong approach where not only supporting the ongoing projects and more or less boot-strapping that up, but you’re bringing up to speed basically the end-user and industry with the technology. There’s a license agreement between the Laboratory and industry right now for the Ganni cycle, it’s being utilized in new plant designs.

Ganni:

What we recognized, although we are going and modifying some existing systems, we cannot do everything ourselves unless we make a basic change at the start. The only way we can influence it is by making the industry recognize and implement it from the start rather than us going and retrofitting and doing half the job. If it is not designed from the beginning with this concept, it will never be as good as it could have been. So JLab licensed that technology to the industry.

Arenius:

We also tried to be good stewards as far as looking at helium gas. Helium gas is becoming scarce commodity in support of research these days, and also for industrial needs as well. Many times we see that people are cut back to 70% of what their real needs are because of the scarcity of the helium. But we recognize that these helium refrigeration systems and smaller experimental stations that you find at many universities have basically once-through type use of helium without recovery. Starting with the SNS project, we demonstrated that it is possible to build these systems with very little helium loss. And so at SNS, besides the improvement of efficiency, the cost of helium replacement due to system leakage was dramatically reduced. Here at Jefferson Lab we probably go through around $400,000 worth of helium a year due to helium losses. Other laboratories and other universities go through up to a million dollars a year. At SNS we show that with a half-scale size of our CHL, but we have the same number of compressors and the cold box, only half the cryo modules, it’s $10,000 a year. And so here at Jefferson Lab we developed a helium purifier. Now in industry there is no company that even looks at developing helium purification system for lab use. It’s something that is needed in order to recycle the helium and reutilize it, but that technology is not a product line of industry whatsoever. So there are components within these refrigeration systems as the laboratories use them or can see cost savings that have to be developed. And so we developed a standard purification system, which was funded by industry to become one of their standard products, so that they could provide that support to other laboratories and the industrial users.

Ganni:

We train our employees to do their graduate research work on systems of need for the cryo community. They include purifier design, standard liquefier capacity and efficiency improvements with industrial funding. Also we developed the new compressor design for NASA and 12GeV funded by NASA. We are developing 4 K - 2 K HX for increasing the 2 K process efficiency, funded by FRIB.

Arenius:

The product was a standard helium refrigerator that is typically used as a refrigerator in a university laboratory. It’s a smaller-scale experimental type refrigeration. But you literally have hundreds of these out there in the world, so there was a need to improve that efficiency.

Ganni:

Actually in 2004, Jefferson Lab started a cryogenics workshop to bring the users awareness of all these aspects of what we learn here and everywhere else to be shared. That workshop was started by Jefferson Lab in 2004.

Arenius:

It’s worldwide, such places as CERN and DESY, as well as here in the U.S., and now in Asia. Every two years we get together to go over the latest where are the end-users? Where are the demands going? What do we need? What problems have we solved? What are the solutions? To share that information.

Ganni:

In addition we also teach cryogenic engineering courses at CEC and other locations including universities. This is another form from our group that we teach, to bring users awareness up.

Westfall:

Okay. Thank you very much. I’m going to turn the tape recorders off.