close
The Wayback Machine - https://web.archive.org/web/20101018180948/http://fixthepumps.blogspot.com/

Fix the pumps

Friday, September 24, 2010

The pins

So we've covered the trench the gates sit in when lowered, the gates themselves, the guide columns, and the seals. Next up - the dogging pins.

As detailed in my post "The Seals," at 17th Street there are provisions to keep the gates in place horizontally. Namely, mating wedges on the needles and the guide columns which force the gates against the back (city) face of the guides (diagram adapted from 17th Street as-built drawing here):

BERJAYA
There is no similar system at the London or Orleans gates, meaning the gates can rattle around in the guides when lowered or raised, as well as when they are seated in the trenches.

But what keeps the gates in place vertically? The answer is dogging pins. Dogging pins are large cylinders slid into circular slots on the needles to restrain them from moving. There are two for every gate section (pictures from USACE Flickr stream here and here):

BERJAYA
BERJAYA
At all three sites, the dogging pins slide into slots near the top of each needle when the needle is placed in the trench (picture here):

BERJAYA
This serves to "lock" the needle in place, as well as letting the operators know that the needle is firmly seated in the trench. The pins are very similar to the lock on a double-hung window; that lock pushes the sashes into the framework of the window, ensuring a tight fit.

There is a minor difference between the 17th Street and London/Orleans designs in how the gates are stored when they are not lowered. At 17th Street, the needles hang off a second set of dogging pins placed at the top of the main framework (from this drawing)...

BERJAYA

Here's a picture of the upper pins and their frameworks at 17th Street. This picture, from the 2007 SAME presentation I cited in my earlier post on the trench, shows the pins before the framework for the winches was added. This would place this photo in the mid-2006 timeframe:

BERJAYA
At London Ave. and Orleans Ave., the dogging pins at the platform level do double duty, supporting the needles in a second set of circular slots (from London Ave. bid drawings):

BERJAYA
As mentioned above, the pins signify to the Corps personnel that the gates are firmly seated in the trench. Remember that the Corps is supposed to be having their diving contractor clearing those trenches regularly, as well as immediately before any closure of the gates during a storm. So if the pins don't go in the slots on the needles, that means the trench is not clean, which is bad.

Here's what the 2009 Operating Manual for the gates says about the pins:
"To close a gate, it will need to be lifted slightly to allow the dogging pins to be removed. A come-along may be needed to remove the dogging pin. Once the dogging pins are removed the winch can lower the gate until the lifting cables go slack. Once the cables are slack the dogging pins should be inserted into the holes near the top of the gate that are meant for locking the gate in the closed position. If the dogging pins can be inserted when the gate is down, this will confirm that the notch in the sill is free from obstructions. The eye bolt may need to be removed from the dogging pin and a sledge hammer used to move the pin into place. This process for closing the gates is identical to that described in subparagraph 3.2.1.2. If gates do not fully close and conditions are no longer suitable for divers, pre-positioned sandbags within reach of the crane shall be utilized to seal gaps."

One can see those pins getting pulled before lowering the gates during the May 27, 2009 media availability in this Times-Picayune video, starting at 0:43:

Testing the Pumps

If one looks at the dogging pins closely, and then considers the lack of horizontal restraint on the gates at London and Orleans, one might wonder, "If the gates are rattling around during a storm, what keeps them from knocking the dogging pins out?"

The answer is a second set of pins. The Corps had the contractors add cotter pins which slide through the eyebolts on the ends of the much bigger dogging pins. These cotter pins are bent pieces of threaded rod which screw into welded nuts. Here's a few pictures (from here, here, and here):

BERJAYA
BERJAYA
BERJAYA
A similar set of second pins is also installed on the dogging pins at 17th Street.

Effectively, the threaded rods are the last line of defense for the gates. That is, the threaded rods restrain the dogging pins, and the dogging pins restrain the gates, and the gates restrain the storm surge. Of course, the gates are pretty heavy, but so were the wall sections that collapsed along the canals. Water can do amazing, scary things.

Monday, August 30, 2010

Quick update (with pretty new trees!)

I thought it was time for a quick summary of my most recent posts.

First, the hydraulic pumps. The hydraulic pumps (54 of them scattered across the three sites - 17th Street, Orleans Avenue, and London Avenue) are rusting to bits, and the Corps is doing next to nothing about it. These pumps are supposed to last another four years, minimum. Here's what two of the pumps looked like after just two years in the water (pictures taken from the 2009 Conhagen pump repair report):

BERJAYA
BERJAYA
BERJAYA
BERJAYA
BERJAYA
BERJAYA
These pictures were taken over a year ago. Imagine what the unrepaired pumps sitting in the same water at the same sites look like now.

My initial four part series on the corrosion crisis detailed the repair effort last year and this year to belatedly address this damage to the pumps, as well as how they got to the conditions above.

The 2009 repairs are covered in "Imminent"
The background is in "How did the pumps get from..."
And the work from this year is detailed in "This year's scramble" and "Worse than previously known"

The repair efforts are underfunded and way too slow to make a difference this hurricane season. Thus, here's the status of the pumps at each site, based on their level of repairs:

BERJAYA
BERJAYA
BERJAYA

I recently updated the repairs. They are stopped on the main 60" pumps at all three sites this hurricane season, but have continued on a few 42" pumps at 17th Street. Specifically, four of the fourteen 115 cubic feet per second pumps (and only those four) mounted under the deck of the 17th Street structure have been under repair for about five months. Those 4 pumps represent just 5% of the overall capacity at 17th Street.

My recent update on the repairs can be found at "No Urgency."

That work did not appear to happening safely, and I described what appear to be possible OSHA violations during the reinstallation of one of the pumps in "Safety first:"

BERJAYA
BERJAYA

I also reported on the 40 oil spills, accounting for thousands of gallons spilled, the Corps has had at the gate structures over the past 4 years. The vast majority of the spills - 29 of them - have not been reported to the Coast Guard as required under federal law. This appears to violate the Clean Water Act, a violation with penalties including prison and tens of thousands of dollars per day each spill goes unreported.

I detailed all those spills and their background in a three post series:

Corps of Oil, Part 1
Corps of Oil, Part 2
Corps of Oil, Part 3

I recently did a follow up on the oil spill story, called "Boom goes the oil," on the spill control technology (i.e. booms) the Corps uses, or doesn't use, as seen earlier in August:

BERJAYA
BERJAYA
BERJAYA
The downstream boom, which is the final boom that prevents oil leaks from escaping out into the lake looked like this:

BERJAYA
All that boom is there because of all the oil leaks, which happen because of all the corrosion. In fact, that is how the Corps determines which pumps to repair - they've gotten so badly corroded that they've released oil into the canals.

Recently, I've started looking at the gates themselves, which are two feet too short. I examined the underwater trench into which the gates settle in "The trench," and raised questions about what condition those trenches are in at 17th Street and Orleans Avenue. The condition is questionable because all the grout found all over the side of the site at 17th Street...

BERJAYA
BERJAYA
...was also squirted into the trenches at 17th Street and Orleans Avenue. In fact, one gate segment got cemented in place. It's shown here shortly after being freed in August, 2006:

BERJAYA
Taking the amount of grout on the gate as a guideline, there was about 3 feet of solidified grout sitting on the bottom of the canal over the trench:

BERJAYA
"The gates" took a look at the gate segments themselves, as well as the columns they slide within. Both diverge significantly from the Corps' own design guidelines, which call for much sturdier structures.

My most recent post, "The seals," examined whether the gates actually are designed to keep out water, and concluded that at two of the sites - London Avenue and Orleans Avenue - they are not. In fact, the seals that should be between the gates and the structure are actually bolted on the outside of the structure, providing no sealing at all:

BERJAYA
Looking closer:

BERJAYA
That allows water to flow around the unsealed gates through gaps:

BERJAYA
BERJAYA
Considering the reliablity of the pumps immediately behind those gates, the system looks less than robust.

With all this - including pumps which are in imminent danger of failure due to systemic rusting - the Corps found $273,000 to plant trees and bushes around all three sites this spring:

BERJAYA
BERJAYA
BERJAYA
BERJAYA
BERJAYA
BERJAYA

The contract went to Perez APC on March 28, 2010 and was likely subcontracted to Rotolo Consultants Inc (RCI) of Slidell, LA. The contract, W912P8-10-P-116, along with all its modifications and the original bid solicitation, can be found here (the Corps has thus far refused to release the drawings).

Why does this matter?
1) $273,000 would repair the corrosion damage to two of the 60" hydraulic pumps, making about 400 cubic feet per second (cfs) of capacity available without fear of failure from corrosion (other causes of failure still exist). There are still at east 35 pump representing over 6000 cfs of flow which have not been touched by the repair effort.

2) All three of the gate structures are scheduled to be torn down in four years when they are replaced with permanent pump stations. These trees and shrubs will undoubtedly come out then too. If this doesn't define waste, I don't know what does.

But, it's all about appearances with the Corps. And as long as the arborvitae and the crape myrtles are doing good, that's all that matters.

Photo credits and outside links for all photos except the trees, which are new in this post, can be found within their respective posts.

Karen Gadbois & Steve Beatty of The Lens and Molly Peterson of Southern California Public Radio contributed to this report.

Saturday, August 28, 2010

The seals

After examining the trenches, the needles, and the guide columns at 17th Street, London Avenue, and Orleans Avenue, it's time to move on to the spaces between all those elements and what should be filling those spaces - seals.

Without sealing, the gate structures will leak storm surge into the canals behind them. What I've found is that at two of the sites - London and Orleans Avenues - that is likely exactly what happens. But we'll begin with the site that actually has seals, 17th Street...

Seals and gaps at 17th Street

[This section of the post will include excerpts from the following as-built drawings for the 17th Street structure:

"Typical Gate Plan, Elevation, Section & Details, drawing 21 of 34"

"Typical Gate Elevation & Details, drawing 22 of 34"

"As built" refers to the final versions of the drawings - they are what is in the field.]

Looking closer at how the needles travel along the guide columns, here's the detail of one of the 17th Street needles between two guides, looking from above:

BERJAYA
There are a lot of design features in these gates that are not found at the Orleans Avenue or London Avenue sites. Let's zoom into the interface between the needles and the guide columns to take a closer look:

BERJAYA
First, there's a pair of welded angles on either side of the front (lake) side that prevents movement of the gates from side to side as they are raised and lowered. Such motion, if it was too large, could lead to the needles getting stuck. The gap between the angle and the guide columns is only a half inch. Here's a photo of the angles:

BERJAYA
Then there's the seals. On the 17th Street needles the seals - known as "j-seals" from their shape (see two photos up) - are installed on the back (city) side of each needle, along the sides and the bottom of each segment. Here they are on the drawings:

BERJAYA
Here's a picture of the seals from the 2007 Corps Infrastructure Conference presentation that I excerpted from earlier:

BERJAYA
It's important for those seals to get compressed between the surfaces of the needles and the guide columns. Otherwise, they're not really sealing. Storm surge alone does not do it. One needs a constant static pressure along the entire length of the needles, which cannot be provided by wind-driven, wave-topped water during a storm. Hurricane storm surge is not the same as a large snow bank.

The 17th Street design accounts for this with mating sets of wedges installed on each needle and guide beam. On the needles, the wedges - which are steel - are installed at the top corners:

BERJAYA
And the mating neoprene wedges on the guide columns are installed at the elevation where the tops of the needles come to rest (photo from SCPR's Molly Peterson as part of her Flickr stream attached to her "Pumps Under Pressure" series from last August):

BERJAYA
When the gates are dropped into place, these wedges force the needles back against the guide beams. Here's details of the wedges from the drawings (I've adapted the drawings to show the wedges "in action"):

BERJAYA
In concert with the mating sloped surfaces between the bottom of the needles and the trench...

BERJAYA
...the needles are pushed back toward the city ("protected") side of the guide columns, scrunching the seals. This assumes the trenches are cleaned out and are in good condition, which is a questionable assumption.

[Note that the design around the seals even includes a small stop block to ensure the seals are not completely crushed when compressed. That block serves a dual purpose of not allowing the gates to swing front-to-back so far toward the city (protected) side that the welded angles on the front cannot do their job preventing side-to-side motion.]

Not everything is great with this design though. There is still some room for the needles to swing front-to-back when being lowered or raised, which can lead to damage to the needles or the guides:

BERJAYA
With W21x101 guide columns, a 15.375" thick needle, a 1.75" thick wedge and a 1.5" stop block, there is 1.13" front to back for the needle to swing. While the needle is swinging in that gap, it could bind in the guide columns, which can cause damage to needles or guides or both.

I realize that without rollers along the guides, some clearance is required. The problem is the lack of rollers, and the resultant swinging of the needles. That movement is most prominent during storm events, when storm surge will be raising the lake level and rainwater could be flowing out to the lake. In that case, the needles will see a "differential head" when they enter the water. That means that there will be water pressure on the city side of the gates which is different than on the lake side, due to the flow of the outgoing rainwater. Having the gates simply swinging in the water, banging against the front and back surfaces of the guide beams (side-to-side movement is limited - but not eliminated - by the angles on the fronts of the needles), and being lowered only by gravity, is really against almost all the design tenets laid out for vertical gates in the Corps' own standards.

In addition, those standards also call for seals on the front, sides and back of each needle. The 17th Street ones only have them on one side. Such a sealing scheme would go hand-in-hand with reinforced guide columns and rollers, topics I discussed in my previous post.

These problems were noted clearly (and admittedly more succinctly) in the Black & Veatch evaluation of the gates:
"most emergency scenarios for closing the gates will require that the gates move through differential head. Gates with rollers are required to move through differential head."

The report goes on to recommend replacement of the guides and needles with a roller gate system. Rollers would keep the needles perfectly aligned rather than banging around against the guidebeams.

Overall - while a more robust design would include reinforced guide columns, rollers along those guide columns, and more sealing surfaces - for what it is, the design of the needles and guides at 17th Street is pretty good.

The story is much different at the other two sites.

Seals and gaps at London and Orleans Avenues

I won't be relying as much on drawings in this section, because unlike at 17th Street there are so few design features to point out on the Orleans and London gates. However, the two as-built drawings for the needles are here:

London Ave. Canal Interim Closure Structure: Needle Gate - Sections & Details, drawing 16 of 34

Orleans Ave. Canal Interim Closure Structure: Needle Gate - Sections & Details, drawing 16 of 26

Just to bottom-line it before getting into the details:

1) The seals are outside of the structure, performing no sealing function at all. There are no seals between the needles and the guides, as there are at 17th Street, likely allowing water through.

2) There are no wedges to force the needles against the guide columns and keep the needles fixed in place when lowered, as there are at 17th Street.

3) There are no angles on the fronts of the needles to prevent side-to-side movement, as there are at 17th Street.

4) The gaps around the needles within the guide columns are huge, meaning the gates are constantly banging around and likely allowing water through.

As the list above demonstrates, the change from 17th Street to the London/Orleans sites is stunning. Beginning with the seals...

The essence of a seal is that it is a compressible material that gets smushed to fill a gap between two adjacent hard surfaces. Think of the seal on a car door, or weatherstripping on a house's front door, or the lid and rubber ring on a mason jar. At London and Orleans Avenues, the seals fail to meet this basic definition, because they are not placed between the two adjacent surfaces of concern, the needles and the guide columns. Instead, they are decoratively attached to the lake side of the guides and are pushed by a single surface of the needles into the lake.

Starting with this Corps photo of the London gates on September 14, 2009:

BERJAYA
The Corps also took a detail shot the same day that really clarifies the location of the seals:

BERJAYA
Yes, they are located outside the needles, bolted to the lakeside outside surface of the guide columns. They've always been there, as this May 27, 2006 shot (via SCPR Flickr) during the construction of the Orleans Avenue gates shows:

BERJAYA
A detail from the applicable London Avenue as-built drawing shows the location when looking from above:

BERJAYA
Let's put this in real world terms. How well do you think the seals on your car doors would do if they were attached to the outside panels of the car? What if you tried to weatherstrip your house's front door by putting flexible edging on the siding? What if you put the lid upside down on your mason jar, with the rubber ring facing outward?

All of these are the equivalent of the "sealing" arrangement at London and Orleans Avenues; that is, there are no seals at London and Orleans. How did this happen?

Going back to the bid drawings for the London Avenue gates, from January, 2006, they show how the interface between the needles and the guide columns was originally designed:

BERJAYA
The top diagram shows the original issued-for-bid design. Unlike at 17th Street, there are no wedges or other mechanisms to secure the needles horizontally, only the sloped surfaces at the bottom of the needle and in the trench (on a remarkable side note, the bid drawings do not show a detail of the needle in the trench).

So the needles, even if properly seated in the trench, would tilt back and forth in the guide columns like a tree getting buffetted by wind. As the bottom diagram shows, when waves come in from the lake and push the needles toward the city, the needles would move away from the "seals," opening a gap, which would close but reappear on the back side of the needle as it saw pressure from outflowing rainwater and turbulence from the Corps' pumps. And since there's no seals between the needles and the guides, and there's nothing like wedges keeping the gates from continually rattling in the guide beams, water would flow around the needles quite easily.

After award of the gates contracts in January, 2006, a modification was made to the London and Orleans designs, ostensibly to remedy this problem. The location of the seals was not changed, which would have been best. Instead, two MC6x12 channels (depth 2.5") and two 1/2" plates would be welded to the lake side of each needle, to provide a kinda-sorta sealing surface against the j-seals. Of course, this would never work:

BERJAYA
One cannot seal against water when a seal is pressed between a single surface and water. As I wrote above, water flow, waves, and storm surge are not like snow - they're not going to provide totally uniform pressure along the entire submerged length of the j-seals, pressing them against the new channels. This is especially true, not only because water doesn't behave that way, but also because there is still nothing in the design (like 17th Street's wedges) to keep the needles from rattling in the guide columns. Their only fixed point of contact is in the trench.

Additionally, at London Avenue there was another change made to the design of the needles and guides that negated the addition of the MC6 channels and the 1/2" plates. Perhaps to accomodate the new channels and plates on the needles, or perhaps for some other unrelated reason, the guides were increased in size. They went from W18x119's (18.97 inches deep) to W21x132's (21.83 inches deep). The needles did not change in size; they were probably already being fabricated at the time.

So, despite an extra 3 inches of thickness at the "sealing surfaces" from the new channels and plates, the even bigger guide columnss resulted in the storm-surge-applied gap between the needles and the j-seals staying about the same as before: 0.8" rather than 0.9". Even more troubling, the larger guides at London Avenue leave a giant 4.5" gap between needle and guide flange, up from just 1.6" in the bid drawings. And worst of all, there is still no seal on the city side of the needles (or any side for that matter), nor is there a mechanism for ensuring the gates do not rattle, opening up these gaps. So this problem was never addressed.

[Note: the drawings for Orleans Avenue followed a slightly different path. In the Orleans bid drawings, there were no MC6 channels or plates to provide a "sealing surface," but the guides were already W21x132's. In the as-built Orleans drawings, the MC6's and plates have been added, making the final Orleans versions exactly the same as the London versions.]

The 4.5" gap between the needles and the front flange of the guides can be seen in this Corps photo taken during the May 27, 2009 hurricane exercise:

BERJAYA
The permanent damage to the "seals" from pushing them outward continuously is also clear in this photo from the same event:

BERJAYA
If one can see gaps between the sealing surfaces and the seals on a sunny day, how many and how big are the gaps along the seals during a design-or-bigger storm event, with the needles rattling around front to back and side to side (they move side to side because there's no angles to stop them, like at 17th Street)? The water would be at least ten feet higher than what is shown in the photo above, with at least two to three foot swells on top of that. Water would be lapping at the top of the gates, if not washing over them. The gaps between the needles and the "seals" would probably be at least an inch, possibly more depending on how much those seals have gotten permanently flexed outward over the past four years.

And it's not like there isn't any guidance out there on how to do this. The Corps' Vertical Lift Gate Engineering Manual says the following in its section on seals:
"Seals can be mounted on the skin plate side or the flange plate side, oriented so that the water pressure is acting on the stem of the J seal increasing the contact pressure of the seal."

That is exactly how the 17th Street seals are oriented and is exactly the opposite of how the London/Orleans seals are installed. There is no way to increase the pressure on the seal when there is only one solid surface pressing on it.

Conclusions

The inadequacy of the seal design at the London and Orleans sites, combined with the sizeable gaps between the needles and the guides at those sites, makes me very wary that the gates there actually fulfill their primary duty of stopping the water. At the very least, the design is ripe for creating conditions where needles get bound up while being lowered or raised. I believe that is what happened during the May 27, 2009 pre-hurricane-season exercise/show for the cameras. From the Corps press release describing that event:
"At around 10 a.m. today, gates at the London Avenue Canal began to close to block the expected surge from entering the canals when a cable broke, keeping one of the 11 gates in the closed position.

'The cable breaking was not part of the exercise, but the team was still prepared and reacted extremely well. They had a spare cable onsite and were able to replace the cable within an hour,' said Emergency Management Chief Mike Stack. 'For a real event, we would have cranes on hand as a backup to raise or lower gates should the need arise.'"

In sum, it appears the London Avenue and the Orleans Avenue gates structures are far more permeable than the Corps would have the public believe. While that is troubling enough, it's more perplexing that these problems have never been corrected in the four years the structures have been in service. The Corps anticipates using these gates at least four more years. I bet they will probably never get fixed, but it makes one wonder how much other work throughout the rest of the system is just for show like the seals at London and Orleans?