Atlantic Council
The Contemporary Challenge of Cost and Program Analysis at the Pentagon
Dr. Jamie Morin,
Director, Cost Assessment and Program Evaluation,
U.S. Department of Defense
Steven Grundman,
M.A. and George Lund Fellow,
Brent Scowcroft Center on International Security, Atlantic Council
General Craig McKinley (Ret.),
President and CEO,
National Defense Industrial Association
Tony Bertuca,
Chief Editor,
Inside the Pentagon
Location: Atlantic Council, Washington, D.C.
Time: 10:30 a.m. EDT
Date: Thursday, September 17, 2015
Transcript By
Superior Transcriptions LLC

GENERAL CRAIG MCKINLEY (RET): Good morning, ladies and gentlemen. I’m Craig McKinley, the president and CEO at National Defense Industrial Association. On behalf of our chairman, Arnold Punaro, who’s with the secretary right now, welcome. What a distinguished group. I know we’re going to have a great lecture today. And it’s my pleasure to be part of the introduction.

We are going to hear from Dr. Jamie Morin, who is a good friend of mine, director of cost assessment and program evaluation. I want to thank Dr. Morin for making his time available, with his busy schedule, today. I also want to thank the leaders of the Atlantic Council for their partnership, and in particular Governor Jon Huntsman, chairman of the Atlantic Council, Fred Kempe, its president and CEO, and Steve Grundman, the M.A. and George Lund fellow. And let me also welcome NDIA board members who are present for this morning’s address, Bill Lynn, Ed Fortunato, Jim McAleese.

An address by the director of CAPE is very timely, given the budgetary challenges we face. CAPE is one of the secretary’s most important staff organizations for asking the key analytical questions about future investments. But if we’re headed – (coughs) – into a year-long CR, that analysis maybe is largely for naught. That’s why NDIA joined with AIA, the Air Force Association, the Navy League, and the National Guard Association on Tuesday in a letter to the leaders of both parties in Congress, asking then to avoid the CR trap.

We call it the CR trap because long-term CRs just lead to more and even longer-term CRs. Freezing your budgets from year to year isn’t a way to run any enterprise, let along the United States of America. Congress can’t let us get caught in the CR trap. Of course, when we escape the CR trap, we also need the help of Congress to lift the Budget Control Act spending caps. We cannot balance the budget using discretionary accounts. You don’t need to be a budget expert like Jamie to see that.

But the secretary is genuinely lucky to have a real budget experts as his director of CAPE. CAPE is an organization with an incredible history that wields as much as power as any office in the Pentagon. For those of you who may not know the history behind the title of today’s address, Now How Much is Enough borrows from a classic book written by the original director of CAPE, then called “Systems Analysis,” Alain Enthoven. Answering that question is CAPE’s key mission. Given the threats we face and the missions we must accomplish, how much capability and budget is enough? And as we figure on how much is enough, what must we cut in order to make room?

On top of those key questions, Congress expanded the organization’s mission in 2009. Now, CAPE is also tasked to ask how much is enough for acquisition programs and produce independent cost estimates so that the secretary and the Congress can make an informed decision before investing taxpayer dollars over many years. The change to independent cost estimating has already contributed greatly to cost realism in acquisition reforms, and better proven program performance.

I look forward to hearing Dr. Morin’s answer to the question, how much is enough? So I will hand the podium to Tony Bertuca, chief editor of Inside the Pentagon, and the media sponsor of today’s event. Tony? (Applause.)

TONY BERTUCA: Good morning. I’m Tony Bertuca, chief editor of Inside the Pentagon, flagship publication of Inside Defense, which will be unveiling an all-new redesign in the next few weeks.

Dr. Morin, the man you’re actually here to see, was confirmed director of Cost Assessment and Program Evaluation on June 25th, 2014. He is responsible for analyzing and evaluating the department’s plans, programs and budgets in relation to U.S. defense objectives, projected threats, allied contributions, estimate cost, and resource constraints. Dr. Morin previously served as assistant secretary of the Air Force for financial management and comptroller. He has also served as acting undersecretary of the Air Force.

From 2003 to 2009, he was the lead analyst on the U.S. Senate Budget Committee for defense intelligence and foreign affairs budgets. Earlier, he worked in the office of the undersecretary of defense for policy and also served as President Obama’s Defense Transition Team. I was also informed this morning that Dr. Morin has a Darth Vader helmet in his office, so I guess that has a lot to do with being a comptroller. So, ladies and gentlemen, please welcome Dr. Morin. (Applause.)

JAMIE MORIN: This is actually not the speech you’re looking for. (Laughter.) Wait, that’s the other side.

Well, ladies and gentlemen, thanks all for coming out. And, Craig, great, great pleasure to see you again and appreciate your leadership at NDIA. General McKinley and I, I think, first had the opportunity to get to know each other and work together when he was in the Air Force programming and strategy shop. That’s quite a few years back now, and obviously went on to be the first four-star chief of the National Guard Bureau, and I think maybe the first Guard officer to be the head of the Air Force Association, speaking volumes to his credibility across a wide range of communities. Very much appreciate the introduction and very much appreciate the support of Inside Defense and Inside the Pentagon for this series. I wish my organizational budget was large enough to afford a subscription. I’m told it’s great. (Laughter.) No, I read your stuff. It’s good.

We are in an extraordinary period for the nation and an extraordinary period for the Department of Defense. It’s an extraordinary opportunity for great analysis to support decision making at every level in the department. Unfortunately, we’re also in an environment where it’s possible that we may find ourselves sliding into a budget environment where no decision making occurs, exactly as Craig laid out. It is a frustrating time, because I think the department is on the verge of making an important shift in where its priorities are and how it resources them, and how it advances the war fighting capability in the United States. And we’re talking about a full-year continuing resolution that would make all of that moot. It would give the department no ability to set into motion the changes that we’ve built an analytic foundation for.

Now, it’s kind of unconventional for me to be here. The CAPE director doesn’t spend a lot of time talking with industry or in front of the public. And there’s some good reasons for that. You know, in CAPE we run on behalf of the secretary, on behalf of the deputy the resource allocation process. We run on behalf of the department as a whole and the acquisition system the independent cost estimating, bringing independent views to the fore there. None of those are generally the sort of thing that you like to do in the public eye, or in – as part of the public discourse. Sort of better neither seen nor heard, except by those who need to.

But while it’s important to give the department space to make the kinds of decisions that we need to make on where to go with quantity versus quality of systems, where to go versus – in terms of force structure versus capability and so forth, we also do it owe it to the broader community that supports the national defense to make clear the foundations on which we’re building the various decisions that the department proposes, because this is democratic system and the people’s representatives have the power of the purse. And the broader public, you know, to the extent we can consistent with protection of national security, should be aware.

So I will admit I’m in the middle of program review right now. And depending on the questions people raise, I may have to demure on some of them to protect the secretary and the chairman and the deputy and the vice’s decision space. But I will try to talk as much as I can to the principles underlying what we’re trying to do. And hopefully anybody who was hoping for a more inside gouge was looking for inside gouge on the Fed rate move, on which I truly can’t provide any insight at all. (Laughter.)

While there are sensitivities, I think it’s a mistake for the director of CAPE not to be out and engaged with industry and with the public on a regular basis. So I’m grateful for the opportunity. If you think about it, certainly Secretary Carter has stressed the importance of a tighter dialogue between the department and the industrial base on which our national military superiority is grounded. He’s stressed that throughout his career in the department. As AT&L, when I first worked with him closely, he said very clearly, you know, a dialogue between industry and the Department of Defense about the future of our industry – our industry – is needed to chart a course together. I thought that was an important statement he made.

And that focus clearly continued in his term as deputy secretary. And now, as the secretary, he’s laying that out over and over again. He’s emphasized, we need to prepare a force for a complex future. He’s emphasized, we need to increase the permeability of our force – civilian, military – that the intellectual basis of the Department of Defense needs to be a more permeable one with more opportunity for flow of ideas in and out so that we’re bringing the best to bear on our problems.

Toward that end, he’s done things like announcing expansion of the SecDef Corporate Fellows Program. That’s a group of military officers, primarily, who I take pains to meet with every year so I can learn from what every one of those classes is learning from many of the companies represented in this room and many other companies completely unrelated to the defense industry. And he’s – the secretary’s been out now a couple of times to Silicon Valley to – in which he’s listening, he’s learning, he’s talking. And it’s really all about capturing the most exciting and transformational and aggressive and useful ideas he possibly can. And that’s both tradition industry partners, but also completely new ones.

So heavy focus for the secretary, and thus I am here. I hope it’s satisfactory for all of you. I will say again, though, I’m sorry to be speaking to you under these conditions, because we are in an environment where the national political dialogue is essentially saying to the department that we don’t care about the strategic judgment of the military and civilian leaders of the Department of Defense, we don’t care about the strategic directions that the Department leadership is trying to take the department in.

And we’re seriously considering, for really the first time in my career in Washington, putting that entire defense establishment – $1/2 trillion a year establishment that touches millions of Americans every day – on a dangerous autopilot. That is – it’s a pretty extraordinary circumstance. You heard the secretary of defense talk about it in some length yesterday at the Air Force Association. We saw that very interesting open letter from the industry associations and others. Continuing resolutions, and particularly a notion of a full-year continuing resolution, bring together just a whole host of challenges for the department.

On the one hand, there’s all the classic challenges of those continuing resolutions that we’ve dealt with every year – the short-term ones – that tells us you can’t start any new programs, you can’t buy more of programs where you were looking to ramp up production, you have your funding stuck in all of the categories and pots that it was in in the previous year. We’ve learned how to deal with those inefficiencies, those programs in the short term. Imagining that extended for a full year is a dramatically different question, though. And many of the management tools that we use to muddle through a continuing resolution go out the window if it’s a full year as opposed to limp through a quarter and then make it well.

But there’s also a – there’s a whole nother piece to that. And I’m going to get to that in a little more depth later. But where I’d really like to begin with today’s discussions is talk about how CAPE and how the analytic community in the Department of Defense can really be in much more of an alliance and partnership relationship with the broader industrial community. And that is maybe something that seems unnatural to folks in this room, because traditionally my team is viewed in a fairly adversarial role, right? We’re the ones that look at the bright ideas that come in and suggest that there might be some holes in them, suggest they might end up costing quite a lot more than initial estimates or glossy brochures believe they – you know, assert they might, suggest that maybe they don’t actually solve all the problems that they’re cut out to, or billed as selling. And that is part of our role.

But it turns out, there’s a lot of good ideas out there. And we’ve only got so much money to spend. DOD’s got, you know, a $1/2 trillion budget. That is a large number. Even in Washington, that’s a large number. It’s an extremely large number. But I have a lot of math Ph.D.s working for me. And they inform me with very high confidence, it is a finite number. (Laughter.) Not all good ideas in the world can be funded in a finite number. So we’ve got to assess how much is enough. We’ve got to weigh the good ideas one against each other.

And what’s most important, is we’ve got to make sure that the good ideas that we settle on we can deliver, because if we don’t deliver them to the war fighter, to the American public, they’re meaningless. They may bring some revenue to the people that are working on them in the short term, but in the long run, to the war fighter, to the American public, they’re meaningless. So that’s a core partnership because people in the defense industry are not in business to deliver failed programs. That’s not good for anybody. We’re in business to deliver actual capability as a team. And that’s where the team is.

Now, our budget is a finite budget. I personally have a lot invested in getting to a place where it’s finitely about $36 billion more than the sequester caps. That’s what the president’s asked for. But even if we get to that level, it is still a finite number. Even if the BCA goes away and we’re able to request as, you know, part of a future budget processes exactly what we think is required to execute the strategy, it is still going to be a finite number. And in the case of any finite number, we need to do the best we possibly can to allocate it and to execute it.

Now, I think that’s a commonsense idea for the American people. I think that’s a commonsense idea for even most of us here in Washington. But where we get into trouble is when we tell ourselves it’s an easy choice today, that there is an easy option, that there’s a simple option. We have an idea, you know, and we think it’s a good idea. And it’s maybe not quite good enough or, more often in terms of the things I see, not yet fully fleshed out, not fully refined. And so we say, well, let’s just start it a little bit. Let’s go, you know, do a feasibility study. Let’s begin with a demonstration. Let’s fund the initial advanced development for the good idea. Let’s start the initial low-rate production for the good idea.

Pretty soon the proverbial camel’s nose is, indeed, under the tent. And it’s not that I don’t like camels, but I prefer to have only invited guests inside in my tent. I say that as a hiker. So it’s definitely for the best. And the tougher the budget environment is, the smaller the tent’s got to be. And so that’s where CAPE comes in. We’re not here just to bash good ideas. We’re here to make sure we all know what we’re getting into and that we maximize the chance of delivery on it. I think that role is incredibly important, and I think it’s all the more important in times of budgetary stringency, because when times are tough we tend to tell ourselves that we can find ways of doing better, that we can find ways of having things cost less, that it’s OK if we lowball a program because we’re going to be able to come up with creativity to make it work in the future.

If you’ve listened much at all to Frank Kendall, you’ve heard his cite a study that was done by a guy named Dave McNicol who used to be the cost director at CAPE. He’s now at IDA. He did a pretty extensive study of cost overruns in acquisition programs. And what he found is that the strongest single predictor of whether a program would overrun and how much it overran was whether it was started in a time of budgetary stringency. It wasn’t is it an aircraft program or a ship program, it wasn’t what service is running is, it wasn’t what acquisition reform regime it was done under. It was, is the budget relatively flush when it starts, in which case it’s more likely to come in at or under cost? Or is the budget stringent – are people feeling restricted when it starts, in which case it’s much more likely to overrun?

He doesn’t – he can’t get to the why exactly with his method, but I think we instinctually have a pretty strong sense that a lot of it has to do with baking excessive optimism into the initial stages, so that we can do all the things we wish we could do. Again, that sounds like CAPE’s role of being the independent cost estimator and speaking truth to power on those things and this is what it actually is likely – most likely to cost to deliver the program, that sounds like the traditional bogeyman role. But I would argue that’s an alliance role because, again, it’s not in anybody’s interest for these programs to fail or to not deliver. In the long run, that is not the right answer.

I would also note that what we see throughout history is as we get on these cycles where we have programs that are failing, in part because they started with unrealistic cost and substance expectations, is we build a vicious cycle where the Congress and the American people lose confidence in our ability as a department, as a defense and national security establishment to deliver. We start hearing about $700 hammers and, pick your story. And it becomes part of a context that undermines support for what is required to defend the nation. We need to avoid that. And we need to avoid especially that vicious cycle.

So we need to acknowledge at the outset, if we’re going to let the camel’s nose under the tent, that we’re going to be prepared for the hump that follows. And need to understand, what’s the size of the hump and what does the hump mean for the joint force? How is it actually going to contribute to our war fighting capability? Now, the good news, from my perspective, is that I’ve got a whole team dedicated to doing that, in all of its difference aspects. I’ve got a team that’s dedicated to getting those cost estimates right at the outset. I’ve got another team that is active in supporting the analysis of alternatives at the outset of the program and ensuring it’s technical sound, and then is involved in the resource allocation, decision support for the secretary and the chairman and the senior leadership of the department as we make the individual programmatic decisions.

I’ve got still another team that actually maintains the authoritative data on all of this so we can take a historical perspective going back to the 1960s, in some cases to 1947, rather than just this is what we all remember from our own personal history. And then I’ve got a team that’s dedicated to breaking us all out of those narrow analytic lenses and getting to the broader picture – an analysis and integration team that looks at whole strategic portfolios, that looks at broad questions of what the department delivers for the money. And that is – that’s an extraordinary privilege for me to be associated with that combined team of folks, and to work with them to drive to the best possible decision making we can make for the nation.

I’ve been grateful to have strong support from the administration. I think, you know, the president viewed – one of his first significant legislative accomplishments was the passage of the Weapons Systems Acquisition Reform Act, that Craig talked about, and that has been in empowering. It’s been empowering in moving analysis left, moving it earlier in schedules, letting us make tough decisions before we have too many sunk costs, letting us set the tone and tenor for programs before they get so far down the road that it’s either politically, economically or militarily infeasible to find alternative options because, of course, once we’ve lost time we can’t get it back. Once we’ve, you know, spent three, four, five, seven years on a non-viable programmatic solution to a military problem, we are now – we would then be up against a wall trying to catch up when we look for an alternative.

But I actually think – and I understand this is in the venue of a lot of criticism about the department, about cost overruns, about individual programs that are still troubled – I actually think we’ve make extraordinary progress. And I think the data supports that. Let me say, I think I see an emerging, and strong, and useful consensus between the independent cost estimators of the department that is indicative of stronger standards, more professionalism and more rigor throughout the process. Prior to the Weapons System Acquisition Reform Act we were seeing differences between the independent estimators in then-PA&E and their counterparts in the services of, on average, 6 to 7 percent in terms of the estimated cost of a program. That number is down now to about two.

Standard setting, consistency, sharing of data, sharing of assumptions, discussion, has really helped us get to much more – much more combined confidence in our estimates. And I think the services has added an enormous amount of rigor. I saw that in my six years with the Air Force, unquestionably. We’re also seeing progress in outcomes. You know, in 2009, the department, they had 10 Nunn-McCurdy breaches. After WSARA we’re down to five in 2011. I think we had two in – last year. And so far this year, I and Frank Kendall, I don’t think we’ve signed out any.

Now, I’m sure there will be breaches in the future. And the goal of the legislation is not to drive to zero, that would mean we were taking too little risk. But that is a pretty clear trend of getting programs started on the right foot and then providing the discipline and support to execute them. It is not an immaterial change. And I think that’s a changed approach and a changed level of rigor that is beneficial to industry as well, for all the reasons I got to before. It keeps us off that negative, vicious cycle.

There is one area that I wanted to highlight today where I really could use industry’s help. And I see a growing partnership here that is helpful, but I wanted to alert a lot of the folks here that are associated with some of the major companies with it. If we’re going to be rigorous in analysis, we need good cost data. We need good data in the progress that’s made in delivering systems and what it’s actually costing. We saw what happened in the 1990s when the department, as part of an acquisition reform effort, decided to drop most cost reporting. We went – I’m specifically talking here about what we call the CSDRs, the cost and software data reports.

That’s the whole basis of the core analytic database for cost analysis in the department. We went from collecting about 1,200 cost reports a year in the late 1980s to under 200 by the end of the ’90s and the early 2000s – dramatically less cost data, and many critical programs with almost none. But then we found when we were trying to start new programs in the 2000s that we didn’t have the analytic foundation on which to assess what they would cost and whether they were likely to deliver. We couldn’t built it up or build it down, either from the engineering level, really, or from the aggregate comparison to historical analogy. I don’t think it’s a surprise in that circumstance that we ended up with program after program that flamed out – whether you’re talking about EFV or, you know, FCS or various satellites where we had costs running out of control. I don’t think that’s an accident. I don’t think that’s coincidence.

We’ve since rebuilt our cost reporting system. And we’re now back to the peaks of data collection in terms of aggregate numbers that we were at in the late 1980s. We pull in every year more reports than we did at that peak, but capturing cost data is always tenuous. It’s tenuous because it is a long-term investment in a short-term world. And it is as much about helping the department understand what we’re getting into when we start the next program, as it is about managing the instant concurrent program. So it is easy to get short-sighted and to pretend we don’t need them. But I think – I think we absolutely do. I think it puts the whole enterprise on a stronger foundation. And I think we’ve got enormous opportunity to make this a more effective effort in its whole.

I wanted to call out specifically the collaboration that’s been ongoing between our cost team and Lockheed Martin on the F-35. On the F-35, our biggest acquisition program, one of our most complex acquisition programs, we now have a contractor that is submitting what we call flex-files. So rather than building an electronic version of a 1960s form, they are handing over a large data feed directly from their enterprise resource planning system, giving us dramatically more insight, reducing the administration burden on them to compile and present the information in a traditional and legacy way, but giving us – but we have the tools to turn into all of the data and looks that we need.

And so we are rolling – we’ve been rolling that out on a pilot basis, and we’re looking to move it – move it forward and get it much more broad. This has been part of a five to six year cost modernization effort that I’m really just jumping on the wave of. But it is extraordinary collaboration between all the military services, the CAPE cost staff, and key industry partners. So we are – and I’m all in favor of reducing bureaucracy and administrative costs, and getting to deeper cost insight, because again I think it lets us make better decisions. I want to hear, and hopefully I can hear from you in the Q&A, about other ideas about how we can be allies, but I think that’s a really important one.

I’d like to pivot now briefly to talk about some of the things that we can do if we work together on getting better outcomes for our nation. And I want to emphasize some of the planning and strategic assessment that went into our current future-year’s defense plan, our current five-year program, and making sure it’s balanced. The 2012 Defense Strategic Guidance and then the 2014 QDR set out what I think was a pretty clear path for the future. It called for a smaller, but more ready and more technologically capable force. And budgets have shrunk and now they’re not growing as fast as we would like, and there’s certainly dramatic risk on that. That means that we as a nation cannot afford to keep force structure that we cannot afford to keep ready.

To do otherwise is to risk a hollow force. And we are hedging against a future that is not – keeping it is hedging against a future that could have very serious consequences for all of the other potential futures. Last year we made some adjustments to the broad plans. So we softened slightly the downsizing ramp for the Army and the Marine Corps. We made some modest adjustments in other services, program, force structure. But we fundamentally remained aligned with the strategic judgment that came out of the 2012 and 2014 reviews. There’s always a lot of reasons and arguments to make for retaining excess force structure, retaining more than you can afford to keep ready, retaining more than you can afford to modernize and have operationally relevant in the future world.

But whenever you do that that comes with a double cost. There’s the straight-up cost to pay the people that are in the force structure. There’s the straight-up cost to equip them. And then there’s the additive cost to actually keep them ready, to exercise them, to have them available. And so we just need to be very careful in approaching these force structure problems, that we understand the ramifications for the nation if we don’t make the adjustments that have been set in train. We have to keep readiness and force structure balanced because – and we can’t just let both of those legs of the three-legged tripod of military effectiveness – which is force structure, modernization, and readiness – we can’t let that tripod get out of balance. And if we are just trading force structure and readiness, that absolutely will happen.

We’ve seen that happen, right? Even as we spent five years living under constrained resources and 14 years in difficult conflicts – first in Afghanistan now Iraq, both continuing in some, way, shape or form – the world – the external world has not stood still. It’s not stood still. The rest of the world and many key nations are modernizing. They’re seeking to invest in critical capabilities, maybe not in the capacity that we have them, but really critical capabilities. And we’ve got focus on staying ahead of those changes if the United States is to deliver the kind of global leadership that the American people have grown to expect. That’s what we have at stake with the budget that’s on the Hill today. That’s what we have at stake.

As the secretary testified today, what makes this budget – the FY ’16 budget – different is the focus that it puts, more so than any over the past decade or so, on modernization. In that 2016 budget we made, I think, key investments to ensure we were putting down-payments down for the future. We focused investments on areas like nuclear deterrence, on space and counter-space, on intelligence, surveillance and reconnaissance, on our ability to operate in anti-access, area-denial environments, on cyber, on guided munitions. These are critical areas where the world is changing and the U.S. needs to act if we’re going to preserve the sort of advantages that we’ve become used to.

But as we make those investments, as we focus resources in those key areas, we need to, and I think we are, doing them in a smart way. There are some cases where we need to invest in absolute leap-ahead technologies that are going to knock people’s socks off. Most of those are things I’m not going to talk about today. But we also know that we have in many cases programs that are delivering capability at a reasonable cost that we can either work to invest more in, so we get more for that – of those known capabilities, or incrementally expand those capabilities.

I look at something like the Virginia-class submarine, right? We had a grand adventure with Seawolf in, you know, an earlier decade. And many people here probably lived through that at a much deeper level than I did. But with Virginia-class, the nation made a less-technically ambitiously, but still developing a world-beating, absolutely top-notch submarine. We’ve now got those producing. We’ve got cost on a very attractive curve there. And so what are we looking at? We’re looking at keeping that production on track and then we are looking at how can you get more out of every one of those submarines.

How can you make targeted investments in the capability resident on each submarine? How can you look at doing things like expanding the submarine with the Virginia payload module to give it more carrying capability so it can put more weapons in the fight if you need it, more other capabilities to the fight if you need it? These are smart, comparatively incremental investments, with huge return. And again, those kind of smart focuses become very hard to sustain if the fiscal environment remains out of whack.

I think we are going to have a solid defense program for our future years, if we’re able to achieve the budget levels we have asked for. I think it’s roughly balanced. There’s always room for improvement, but I think it’s roughly balanced. I don’t think it’s driven by pipe dream, but by a serious strategic consideration of the joint force needs to be able to do and what targeted investments we can make to reduce the risk that it faces in doing that. But all of what I’ve just said is dependent on breaking out of this cycle of uncertainty and breaking out of this cycle of resource craziness. It’s dependent on the Congress reaching a deal on a budget. It’s dependent on building some level of consensus in this town on a way forward.

We’re in this extraordinarily odd position right now where everybody appears to agree across wide swaths of the government that sequester-level resources aren’t adequate to do the work that the nation needs done. And yet, we’re unable to find a way out of the BCA trap. And now we’re adding this added question, which is: If we can’t agree on the tenets of a budget at the lower fiscal level – and for good reason, right, the choices involved are unattractive to the nation. As Craig said, you know, balancing the fiscal purse of the nation on the backs of the discretionary budget is mathematically not sustainable. It’s simply not sustainable. We’ve got to find a way out of the trap.

And I will assert and hopefully explain exactly why a short-term CR morphing into a full-year CR is absolutely the wrong way to try and find a way out of that trap. We know – you know, everybody here known the sort of recent blow-by-blow, and the dilemmas on the Hill. But a path out of that dilemma needs to be found and it needs to be sustainable path. That’s why this OCO gimmick that’s on the – you know, on the table, as was discussed a lot this summer, is the wrong answer for the country. I’m speaking here at the programmer, right, from the Department of Defense.

My job is to look five years into the future and look at the force that we will produce with resources over those five years and what it will mean for the next 30 or so years. One-year patches don’t let us get after the sort of investments we need, and particularly in the modernization programs, in order to create a dominant joint force. They simply don’t. We can’t make long-term investments in capability areas on the strength of one-year patches. And that’s what’s on the table.

And we certainly can’t make those investments in the context of a full-year continuing resolution that just tells us: Go do everything you were doing last year. If you think last year was satisfactory across of the federal government, then I guess that’s your answer. But I can tell you in the national security realm that I look at really closely, it’s definitely not. And I have a strong suspicion it’s not in all of the other areas that I am not, you know, challenged to manage on a daily basis. It’s a road to nowhere, just like the secretary said.

The key things to think through are, A, the math, right? The president asked for a $38 billion increase in defense resources in 2016 over what we had in 2015. A CR gives us none of that. Those resources are overwhelming – those additive resources were overwhelmingly focused on getting our modernization back on track. We had covered those – you know, the bulk of the BCA reductions in the modernization accounts, because the other ones were sticky. You can’t change your personnel numbers radically overnight. Your cost to maintain and sustain your force can only be adjusted so much without going hollow. Modernization bears the burden. If we’re going to break out of that trap, we’ve got to get to a fiscal solution. We’ve got to get to an agreement. We’ve got to get to something that provides stability. A CR just doesn’t do that.

So there’s a lot of individual programmatic specifics. The secretary talked about a few of them yesterday. You’ve heard some of the services starting to talk about them. The obvious ones are in the biggest programs. F-35 is scheduled to ramp up this year. Obviously can’t happen under a full-year CR. We’re going to step forward on another – many other major programs. Navy’s ready to step forward on CVN-80. Can’t do that. But even if you step aside from the individual programs, the aggregates are astounding.

The Air Force was projected to begin, and the budget request, the money needed to begin, and ramp some critical recapitalization – their tanker program, their bomber program, F-35 as I mentioned. As a result, they were working toward a 19 percent increase in their procurement account from FY ’15 to FY ’16. In a CR, none of that happens. You know, we can argue about any individual specific programmatic, but that’s a long-deferred modernization effort. Is the nation ready to sign up for pushing that off another several years? From a military perspective, from a Department of Defense perspective, the answer is no. The answer is no.

I feel like this is kind of a depressing way to end this speech. (Laughter.) But I actually fundamentally am an optimist. And I think there are kernels of consensus building, A, on the need to get out of the trap and, B, on some ways to get out of it. I’d like to see those reinforced. It’s going to be hard, but it is extraordinarily important that our defense establishment is built on a firm analytic foundation that is aimed at getting the maximum combat capability out of every taxpayer dollar, and that is aimed at aligning to the world situation that we face. I think we’ve got an opportunity to do that. I hope we can build the consensus in Washington that is necessary do that. And you certainly have it from your Pentagon team, that given the opportunity we will make every effort within our power to deliver exactly what the American service member and taxpayer has every reason to expect from us.

So with that, I would love to take questions. (Applause.) Do you want to sit down?

STEVEN GRUNDMAN: Take that seat right there and we’ll have a conversation.

MR. MORIN: I have slightly parched myself, so you’ll pardon me.

MR. GRUNDMAN: Yeah, sure, by all means take a second.

I’ll offer some preliminaries, one of which is to reiterate that this entire event is on the record. So when they do come to call on you, I would appreciate you identifying yourself and your affiliation, and also waiting for one of our staff to bring you a microphone before you do that.

Well, this is really terrific. That was a – that was an excellent address for those of us who care about the inside baseball of the Pentagon and appreciate its importance. There are many alumnus – alumni of PA&E, systems analysis, CAPE – among them myself, yes, thank you very much – in the room who do appreciate –

MR. MORIN: Now nobody likes you.

MR. GRUNDMAN: – the significance – (laughs) – no, that explains it all right there, doesn’t it – of some of what sounds perhaps in a larger context like inside baseball. So it’s a pleasure and a privilege to have you here, and I appreciate it.

I have a few questions of my own. We have in total about 25 minutes to have a discussion. I assure you all I’ll reserve at least 15 of those for questions from the audience. But let me – let me take on a couple of my own.

And sticking with the inside baseball theme, just for starters, I will not ask you reveal the pre-decisional choices that you are working through with respect to the FY ’17 budget right now, but I would ask you to talk about the process, the calendar. Again, that is – may sound arcane, but it’s important. And it also – it also changes the rhythm in the building. So maybe just using FY ’17 build as an example, what does the calendar look like? Where are you now? How does it come to rest, et cetera?

MR. MORIN: Sure. I can talk a little bit about that. A couple things I’d emphasize. First of all, that the secretary and the Deputy Secretary Bob Work are very focused on building a solid and reliable analytic framework for the decisions they make. They understand the gravity of the choices that they face. And they believe that there are opportunities to tighten our alignment with the strategy, and to deliver more for what we’ve got. But if we’re going to do that, we’ve got to make sure we’re maturing decisions, maturing issues and allowing the best possible judgement and discussion among the senior leadership.

So toward that end, Bob Work set in motion a reboot, if you will of the planning, programming, budgeting and execution process. We took some initial steps last year, but we’ve taken further steps this year to pull to the left a lot of the Pentagon – the classic Pentagon deliverables. So if you think back to your time in the department, and I see former deputy secretary and former director of PA&E, Bill Lynn, has come in. Back to Bill’s time in the Pentagon on his earlier tours – you were a special assistant?


MR. MORIN: How did you recover from that? (Laughter.)

MR. GRUNDMAN: We go on. But go on. (Laughter.)

MR. MORIN: If you look back to that time, we typically spent the summer in an OSD review. Through the period of the wars, a lot of that slipped further and further to the right. And we came to talk about the fall review, and sometimes the late-fall review. The deputy, with the support of then-Secretary Hagel and continues to work with Secretary Carter, is trying to restore some of that process discipline, in part to give the comptrollers a fair shot, and give them a chance to smooth out the rough edges of the broad programmatic structure that we build through the department’s decision making process. So that’s going on.

There’s also a concerted effort to really construct piece by piece that analytic foundation. And so you saw the department spend much of the spring and the early part of the summer on what we call strategic portfolio reviews. And those strategic portfolio reviews focus on a handful, a half-dozen or so, key areas where either the leadership believes there may be need for some additional resources but they want to understand what they’re getting for it, or they think maybe we’re a little out of balance and not – don’t have the portfolio as a whole delivering the right capability mix.

And so my team together in each case, with one other key OSD or joint staff principle, or in one case service player, put together analytic teams to really deeply dive into those. And those have all been brought to the department leadership with a series of recommendations that are now feeding or programmatic evaluation. So those aren’t, of course, the only areas that we’ll be spending time on through the fall, but those are the ones that we’ve identified as areas of maximum opportunity. And you heard me hint at sort of some of those topic areas in my remarks, but they’re the areas you would expect the department to be focused on given the global threat environment and some of the operational developments we’ve seen.

MR. GRUNDMAN: OK. The other question I’d wanted to ask picks up on comments you made about – well, I put it this way. You know, I heard you say, and as you’ve alluded I know from personal experience, that in a larger sense the job of CAPE in the program evaluation dimension is to create balance and coherence in and among the different pieces of the budget and program.

You talked about a couple of them – which to say I heard you just say force structure is a little long and we’d like it to be shorter, metaphorically speaking. Modernization has gotten a little thin and we’d like to stiffen it up, starting in 2016. You didn’t so much talk about, except glancingly, O&M, which of course has goodness in it, what we call readiness, and it’s got less goodness in it, which we might call infrastructure. How’s the balance and composition of the O&M account? What’s the work to be done with O&M?

MR. MORIN: Sure. So you were, I think, also a deputy undersecretary for installations.

MR. GRUNDMAN: Why do you keep – (laughter) – alluding back to this? Yes – BRAC, outsourcing, all these negative wedges we were working off.

MR. MORIN: So I referred briefly to the department’s broader reform agenda in my remarks. And it’s clear than an important part of that is getting at our facilities overhang. We also understand that that has been a politically difficult agenda item for the Congress for many years. I frankly was hoping that this would be a good year for the discussion, and it doesn’t seem to have turned out to be one, although I do sense indications from some of the statesmen on the Hill that there is a growing sense of the necessity there.

A lot stems from that fundamental question about how many places in the U.S. Department of Defense going to be at home and abroad? We’ve made some significant changes in our abroad structure. And as you would expect, there’s plusses and minuses as we adapt to the world’s strategic environment. We haven’t made a lot of shifts in the last decade in the at-home structure. And it is not – BRAC is not a panacea for all of the department’s challenges, but it is quite surprising, once you dig into it, how much of the structure stems from that question of how many places are you going to be?

This came home to me very clearly during my time with the Air Force, where we often referred to sort of the open-the-door costs of a base as being 800-900 people. To provide all of the sort of one-size functions that have to be there. And those costs were roughly the same, whether it was a base with a single squadron or single wing or multiple. So there’s a non-linearity here that the department has to grapple with. And that underscores some of the estimates that you’ve seen out there about 20 to 30 percent overhang.

Now, there are other things the department can do to get after its O&M costs. Some of them involve making sensible choices in the development and manning and operation of our systems. CAPE not too many years ago set up an operations and support cost estimating directorate, especially because there was this sense that a larger and larger share of the department’s costs were eaten up in sustaining the existing weapons systems.

And if we didn’t have a clean eye on both the levers that were available to manage it to adjust that and what the impacts would be of the next generation of weapon systems on our long-term O&S, then we would find ourselves in an even more strained situation for generating necessary modernizing resources in the future. So that is – we’re building, again, an increasingly strong analytic foundation for that kind of decision making, which is really important. It gets to things like concepts of how you man your systems, what are your maintenance constructs for them, and how do you locate them? All of these are incremental choices that are made over time that in total can add to either affordable or unaffordable portfolios of systems.

We also can focus on other cost areas, like IT, like administrative structure. You’ve – we get a lot of – a lot of tension, particularly among the staffs in the building, to the successive rounds of efforts to prune the management overhead of the department. And that is an absolutely necessary step, but of course one we have to implement in a sensible and careful what so that we don’t end up making bad decisions or making grievous errors in the – you know, in how we fight a war or what we buy, because we didn’t have the right talent helping us make those choices out the outset.

But that’s the line that I think Dr. Carter’s trying to set. We’ve been working closely with the committees. One of the – you know, the committees have laid out various targets for us on headquarters reduction. And I think we’re going to take on a pretty substantial effort along those lines, as we have been really ever since Secretary Gates’ tenure, and I suspect ever since 1947.

MR. GRUNDMAN: Right. My last question before we turn to questions from the audience, I want to go back and see if I could ask you to put a sharper point on your overture to industry’s costs analysts. Actually, mines a two-party question. One is, how do you regard or what might you be able to say about the capability, the capacity of industry in cost analysis? And then secondly, I wanted you to put a finer point on what you were asking for. You alluded to the F-35, this data connection that you have with F-35. Is there a more general call to industry that I could ask you to sharpen?

MR. MORIN: Yeah, so I think my – I think my outreach is really specifically as much to the folks in industry managing P&L, and not so much producing – certainly we all have the efforts to produce specific documents and data. But if you’re managing a P&L, you’re doing a data-rich effort to deliver reduced cost and increased revenue. And there’s a lot of visibility required to make smart decision in that world. And what we are finding is that there’s opportunities for a common operating picture, right, replacing flat-file CSDRs with flex-files and getting ERP feeds to answer the questions is really getting to more of a common operating picture for government and business, which builds a trust relationship in a way that, you know, a pencil-whipped form doesn’t. So that’s a specific area I’m focused on.

But I also think there’s just room for enhanced dialogue about the kind of bright ideas that I talked about earlier. The Pentagon, the five-sided puzzle palace, is not the repository of all wisdom in the world. I mean, it may be 95 percent, but it’s not all. (Laughter.) And so we need that, right? There’s value to that. Now, I recognize that part of what I will get when I’m talking with people who are in business is marketing. And that’s fine. That’s fine. I mean, I expect that. But I think that that can be the start of conversation that can be enormously valuable. So I am open to conversation on the technical matter of cost reporting. I’m open to conversation on what are the right strategic and operational concepts for the U.S. military to be pursuing to ensure enduring dominance.

And I’m interested in dialogue also about things like what the department can do to improve its management and internal efficiency. I mean, oftentimes industry is on the tail end of the whole wagging dog. And there’s opportunity that can be surfaced, we just need to make sure we’re picking the right ones, right? The 1990s acquisition reform we took out cost by allowing less reporting on cost, and it turned out, in my view, we started making bad decisions as a result. And that was not an efficiency. That was a penny-wise, pound foolish kind of thing. But reasonable people can differ.

MR. GRUNDMAN: Well, that is interesting. Maybe, Craig, the trade associations, maybe even some think tanks could facilitate the dialogue. Very good. All right, as I said, I’m welcoming questions from the audience. I’ll start with the gentleman right along the row right there.

Q: Good morning or afternoon. My name is Christopher Dujardin (sp). I’m from the Defense Intelligence Agency.

And my question is, you alluded to just now about the O&M challenges. So what is DOD’s intention for manpower challenge, specifically the use of civilian and military personnel versus private contractors for nearly everything that DOD does?

MR. MORIN: So my view is that we have had a – we’ve had a particular challenge in managing all of our sources of human capital because we’ve often been subjecting it to multiple constraints. I’m coming at this a little bit like an engineer. I’m actually a political scientist, but I’m coming at this a little like an engineer, in that you constrain a system and you constrain your solution set in a variety of ways whenever you set yourself a problem. Ultimately, we need a human capital strategy across the department that gets the work we need.

And there are various sources for it, right? There’s direct employment of military members to perform a task. There’s direct employment of government civilians to perform a task. There’s intergovernmental relationships or loans or services between government agencies. There’s direct contracting out for individual talent. There’s contracting out for specific services. We have a whole portfolio. We have had, as a result of an accretion of individual decisions over time, a whole series of different constraints put on there. And, you know, right now we’re in a dialogue about what’s the right now number of headquarters full-time equivalents, OK? Got it.

The real question is, how do you deliver an efficient and effective defense for the nation within an affordable price? And so in an ideal world, I’d like to see all those individual constraints rolled back and we focus on how do we deliver and you have to address short term, medium term, long term, right? There may be thing that would be more efficient to hire a government civilian to do, but you only need the skill for three or four years and you have no idea what you would do with the person afterwards, so maybe you want to go a different path. These are complex, but they’re not actually that complex set of decisions.

So ideally I’d like to clear that all up and just say we need to reduce our spending in these areas, let’s figure out the best way to do it. That is not a practical reality because we’ve had an accretion of decisions, an accretion of legislation. We have numerous statutory decisions in this area that we have to comply with. So it becomes a much more complex, over-constrained problem. I kind of take that now as reality. My view is, we have excellent people coming from all of those communities. They’re doing good work for the country. We’re probably going to be doing a little less in almost all of those. And we want to make sure we don’t make short-sighted decisions. Within that, I’m open to a wide range of solutions.

MR. GRUNDMAN: OK. I’m going to take this question here, and then after him we’ll come to Tony Bertuca.

Q: Hi. I’m Trip Barber from Systems Planning and Analysis. Previously 10 years the Navy’s chief analyst.

In support of your role as the Department of Defense’s chief analyst in putting the department on a sound analytic foundation, where do you find that there are deficiencies in the capabilities and the tools to actually do the analysis to look at these key issues? Where are those areas of capability shortfall in analysis?

MR. MORIN: That’s an excellent, excellent question. Where we’ve been focusing our effort on kind of closing both human capital and toolset gaps has been highly focused on what I would call sort of mission-oriented modeling, where we look at effectiveness of individual platforms in the context of a mission engagement. Not in the context of a giant world simulation, but in the context of sort of a manageable and understandable piece of military conflict. That’s been a focus area for CAPE for the last few years. It is – it gets to a little bit more bite-sized sets. And it requires a lot of technical strength. It requires a lot of engineering and scientific strength. But that’s a focus area.

I do think it is always hard – sort of shifting hats to more of the broad programmatics – it is very hard for the senior leadership of the Department of Defense to understand the relative balance of risk across wildly different parts of the portfolio. This is particularly true when sort of comparing tail and tooth, right? We know we need a level of tail or logistics support, but how do we really effectively trade that off with, you know, buying a few more trigger-pullers or another fighter? Those are very, very hard choices. They’re not going to be solved by a massive computer model, but there’s probably things we can do to better visualize those trades, to better encapsulate them that can help us.

I see a lot of, you know, interesting insights from kind of companies in the data space. We’ve not always had a lot of luck in translating some of the stuff that’s emerging into the private sector into the kind of decision modeling that occurs in the Department of Defense, largely due to the difficult of agreeing on your output, right? Back on with the economic hat, UDLs (ph) for a private company can usually be pretty effectively represented in terms of some sort of discount in cash flow or return on investment. UDLs (ph) for a military organization are deterring, and if you fail to deter, fighting and winning the nation’s wars. And it gets a lot messier measuring. And so help with that is huge, but that’s an enormously hard task.

MR. GRUNDMAN: Please. Thank you.

MR. BERTUCA: Thank you. To the Air Force. The Air Force –

MR. GRUNDMAN: Please identify yourself.

MR. BERTUCA: Oh, Tony Bertuca, Inside Defense.

MR. GRUNDMAN: Thank you.

MR. BERTUCA: The Air Force recently had a high-profile cost estimation reporting error. They say the estimation’s right, but they reported it inaccurately to Congress on the bomber. What does CAPE make of that situation? And also, more importantly, does CAPE agree with the Air Force’s estimate that the bomber could be less than 550 million (dollars) per copy?

MR. MORIN: So I will duck the second half, just very explicitly – not the time to comment on that. On the reporting issue, I haven’t done a personal deep dive, but the look that my team have done supports what the Air Force said, is that it was essentially a clerical error, that they had an estimate and they – in translating it into a budget-based document they goofed. I will say that, you know, my staff was intimately involved in the development of the estimate some years back.

And, you know, we provide oversight to the program. Neither we nor anybody else apparently caught the error in the report which was, you know, a report on nuclear forces, it was not a traditional budget document, which is probably – possibly why it didn’t get the scrutiny. I believe the secretary of the Air Force is doing a little bit more of a deep dive into that, and I’d defer to her for further comment. But again, we haven’t seen anything that suggests anything nefarious. It was just – nor anything that suggests any fundamental change in the estimate. It was just a clerical mistake.

MR. GRUNDMAN: If we can be quick, I’m going to take two more questions before we wrap up. I’ll start there and then I have a question here after him. I suppose if we’re quick we could get three in, but let’s at least get two.

Q: Colin Clark, Breaking Defense.

Jamie, you’re relying a lot on fixed price contracts for a number of your major programs. The RAND guys and a number of your own people over the years have said fixed-price contracts don’t really work, they don’t really contain costs. What has changed? And what gives you confidence or not that this approach will be effective?

MR. MORIN: So, Colin, I think your statement’s a little too bald. I think our view is that fixed price contracts are appropriate to certain types of acquisition. And there is – there is skepticism to many camps to fixed-price development contracts on complicated development – A-12 was one of the classics that’s been written up many times. And getting contract type right is one of the most important things the acquisition community does.

We, as CAPE, in building independent cost estimates and assessing progress against them as we get later into the life of programs, we will look at contract type, and particularly on production contract, you know, we’ll look – if you’ve got a fixed price of some type, we will say, OK, typically we in a contract of this complexity, of this duration and so forth, we will expect to see a certain amount of engineering change, we’ll expect to see – or a change of quantity orders that will result in adjustments to it. And we’ll adjust up an estimate. We’re not going to just take the price tag on the fixed-price and say, yep, that’s what it’s going to be, because we know historically that’s not how it ends up.

But there is no question that fixed-price contracting can contain government costs when it’s used in the appropriate manner. And I think as we’ve gone through particularly this iterative better buying power effort – and we’re onto 3.0 now – a little more of the nuance is creeping back in and making sure we’re not sending too blunt of signals, that this is one-size fits all. It’s not. But unquestionably, there are cases where fixed-price contracting can contain government costs. We’ve seen that. Unquestionably, there are cases where it fails to constrain it because of the type of program or the way we managed it.

Q: Can you give an example of where it’s worked before?

MR. MORIN: Well, I would – (chuckles) – I would point right now to the fact that the government exposure on the tanker contract has remained limited. Now, that was a fixed-price development, but it was on a fairly technically modest program. I mean, CAPE assessed when that was signed that Boeing was going to lose money. And they did.

MR. GRUNDMAN: There’s a question right here, please.

Q: Thanks. Good morning, Dr. Morin. Mike Winters, Mission Analytics, LLC.

You talked at length about the excellent work happening with the F-35, that Lockheed is able to deliver flex-files instead of flat-files to bring in, you know, essentially, I guess, maybe live data – if you could explain that a little. Although I would say many of us are happy to just get a flat-file, instead of the desktop-created, you know, PDFs and other non-repeatable ways –

MR. MORIN: Even that would be a step, huh?

Q: But the real question is, could this serve as a model for FIAR, you know, Financial Improvement Audit Readiness, how, you know, different financial systems and the feeder systems and apparently the AQ systems could link up and get us to a more quick answer in that arena?

MR. MORIN: Oh, interesting and provocative question. I haven’t fully thought that through. Obviously –

MR. GRUNDMAN: Actually, could you translate the question a little bit for the broader audience, now that you’ve found it so interesting? (Laughter.)

MR. MORIN: Yeah.

MR. GRUNDMAN: Because I’d like to find it interesting too, but I don’t know why I should. (Laughter.)

MR. MORIN: Wow. You’re going to get lots of attendees at your next – (laughter) –

MR. GRUNDMAN: That’s right. This is CAPE, after all. We got to do this.

MR. MORIN: Yeah. So what he’s juxtaposing is our collection of execution cost data from industry, actual costs incurred on programs, with the Financial Improvement and Audit Readiness effort of the department, which is to get to reliable, consistent, traceable information on the DOD internal costs, which of course include the dollars we pay out to the contractor.

Typically for audit readiness, it is sufficient for the auditor to be able to say, company X sent an invoice to the government for the following period of performance on contract Y for dollar amount Z. The government appropriately assessed that it had received the period of performance and the good in question, and all of the – you know, the right receipts are signed and certified by the appropriate officials. At that point, the auditor’s scrutiny will typically end because that’s an arm’s length commercial relationship. It gets more complicated in certain cases. But the big picture, that’s the case.

It’s unquestionable true that DOD is learning from industry as DOD implements our own internal ERP systems. And for better or for worse, we’re learning – we’re learning from industry. We’ve all had hiccups in those kinds of developments, and DOD is not done with them. We’re about a generation behind industry on that. So that’s kind of the broader context.

MR. GRUNDMAN: Thank you. Yeah.

We’re going to – we’re going to put a wrap on it right there. I will, Jamie, before I thank you, give you a last word if you have one, or?

MR. MORIN: Let me say, this is a really distinguished audience, a lot of folks I know. And I’m flattered that you would come out. I hope there’s been some useful insight. For those of you that are in industry, the department appreciates the support you’re providing. We understand it is a partnership. It’s not traditional, you know, economics 101. We’ve got a monopolist in some of our industries. And we’ve got a monopsonist, for those of you that studied enough econ, on the buying side in government in many of these. So it requires partnership. We’ve got to do it in always an ethical and legal way. But there is an opportunity to build a better, more effective national defense by maintaining dialogue. I really appreciate hearing from folks here.

MR. GRUNDMAN: Well, I thank you very much for coming. You know, while I’m listening to this, I am reminded – or am prompted to make the following observation. You know, a lot of times, particularly when something bad happens, a program is overrun or off its schedule or something like that, you know, there is a reflexive response either popularly or maybe on the Hill to, you know, what were they thinking? How could they have made such a bald and it turns out to be a bad error?

When in fact, as I think, Jamie, your presence and comments here today suggest, there is an enormous amount of care and thought and analysis at a fairly fine level of detail that goes into things. And, yeah, things do go wrong, because it is a wild dynamic, often working against some imposed constraints within which you have to make these choices. But it’s not because there wasn’t a lot of care and thought and intelligence put into the choices, which you have here demonstrated today. Thank you very much.

MR. MORIN: That’s very flattering. Sometimes we do just screw up. (Laughter.) But we try to do our best. (Applause.)