Yes, I have ignored the client’s expectations here, but I do believe that it is possible to address the way we work with clients, even if that means we lose some.

I think your list of the client’s planning cycle is as good a place to start as any: isn’t that just another waterfall process?

I wonder whether it’s possible to have a cost overview for a year which has next to nothing to do with the reality, and within that overview, projects can be identified and scope/estimates worked out on an ongoing basis. For example, doesn’t a client have an annual budget, which he then gets to spend on projects of his choice which maximise his return? No one will hold him to ransom if he spends his budget and delivers, but the way he’s done it is different to his plan. Or am I being naive?

The ‘average’ marketing client is used to agreeing how much it will cost to get a particular thing done. They structure their business around it. Indeed, the entire business works from a yearly planning cycle that goes something along the lines of:
1. Define objectives for year
2. Work out activities to do them
3. Estimate cost of activities
4. Submit budget to board for approval
5. Start specifying

The client approaches an agency (or multiple if it is a pitch), with the work scope, the budget, the expectation, etc already set. Sooner or later (depending on agency structure) a Project Manager is introduced who’s job is to deliver the expectation, for the budget of the scope, and they haven’t even done the scoping part yet! So, the PM passes those same problems down the line.

While, given the budget, the PM would end up with a better result for the client using the approach you suggest, the problem is that they haven’t been setup by the agency or the client to do that. And most digital agencies are trying to get their clients to understand digital, let alone explain to them that the entire way they approach it is flawed. And it would take a very ballsy PM indeed tackle that problem as they would be having to change the attitude of the client, the agency and the way they conduct business with each other.

So spare a thought for your poor PM, it’s not their fault! (Although, they could take the risk on personally, run the expectations of the client waterfall, use an agile approach with the internal team and know the end result would be fantastic. But that is a huge leap and puts that persons job on the line if there isn’t buy in to the approach from senior management. Which we can assume their isn’t due to things being the way I said they are in previous paragraphs!).