How to measure the creative workers' output who are not filling in the timesheet?

Question

Some Background:

I'm doing off-the-shelf software ( desktop apps).

The Problem:

I am leading a group of 10 developers+designers ( basically all the knowledge workers type) and I give them quite a lot of leeway in doing their job. I don't enforce office hours and I don't glue them to their chairs for 8 hours a day, I don't look over their shoulders, I don't enforce dress code etc etc.

However, I do

1. Define Road Map for my product under development using Redmine. And I define the tasks that should go into each Road Map. The date for a Road Map is fixed ( Usually on a monthly basis) and usually, One sprint per Road Map.
2. After that, I assign the tasks to the developers and ask them to provide estimation for each task. After looking at their estimation, I will either take out or put in more tasks for a particular sprint so that they don't over-commit and can't deliver in the end.
3. I ask them to fill in the amount of time worked on a specific task everyday and the percentage of the progress so that I can keep track of the progress.
4. I would review their progresses at the beginning of every week to see whether I need to move some tasks to the next sprint.

That's it, no standup meeting, no daily monitoring etc. I fully trust that my developers are honest in their time estimation and working time/progress reporting.

As a developer myself, I understand the tendency of the developers to overestimate their ability and the very real possibility ( near certainty actually) that software projects do delay unless prevented by supernatural forces, that's why I don't strictly hold them accountable for unable to deliver all of the tasks they commit at the beginning of a sprint.

Everything sounds pretty good from a developer-centric point of view, right? I am not the pointy-hair manager as described in this question! This should ensure best working relationship between the employers and knowledge workers, no?

The reality is that, some of the developers have a hard time to

1. Complete all the tasks defined in a Road Map.
2. Fail to provide time estimation for all the tasks assigned to them, despite my repeat probing
3. Fail to consistently fill in the amount of time worked on a specific task. They will update the redmine task, but that is when they are done with it. Usually, they are late in finishing a task-- something that always happen in software development.
4. And despite repeated probing, they still fail to consistently fill in the amount of time worked everyday.
5. Usually, they are able to complete only 50-60% of the predefined tasks for a sprint.
6. I do talk to them when I see them fail to do 2) --4), but they keep on repeating the same mistakes.

I could have imagined the worst and assume that their inability to complete all tasks and to provide daily progress consistently as a sign that they are slacking off. At least this is my visceral feeling. But I want to be a more enlightened boss; I want to give them the benefit of doubt. As I review their timesheet on redmine everyday, my heart sinks if the timesheet isn't updated. But still, I would like to believe that the developers are doing their job and they are just forgetting to update the timesheet (ie, I trust them, I want to!). Nonetheless, the inability of the programmers to clearly indicate their progress ( or the lackof) to me strains the emotional part of me.

What should I do in this case? On one hand I understand that the amount of time spent on a task/ the ability to finish all tasks during a sprint doesn't really correlate with a creative worker's true output. But on the other hand, I would need a way to really understand the creative worker's true output and to reward them appropriately.

How to measure the creative workers' output who are not filling in the timesheet?

Clarification

The reason I ask them to fill in spent time is for their own good, so that they can get better and better at estimation time by comparing their estimated time and their spent time. But, let's say if they are too busy to put effort in improving the their estimation ability, since the data is there, we can use Redmine plugins to enable us to better predict when the software will finish ( think of EBS in fogbugz).It's not for the purpose of controlling them.

Secondly, Redmine allows one to indicate the progress (% done), which is why I would want my devs to update the progress done. I would want them to indicate the amount of time left for a task, but this is not really possible via current system.

Thirdly, as to why I want them to fill in timesheet and estimate the progress? The reason is that our software needs to have a Road Map, so that we know by when we can deliver what. We need to be responsible to our customers and clients and by responsible, I mean, we need to be able to tell them the feature they requested can be implemented by what date.

Fourth, When I talk about reward after completing a project, I am not talking about using monetary stick to move them towards my goal, but rather, i want to know which developers are more capable and therefore can be promoted etc. I know it's pretty hard to do the whole Rewarding System correctly without impacting the morale of the company, but you gotta have a system to recognize the good ones from the bad ones. At the very least, I need to know how to pay everyone fairly. If you treat everyone the same ( the good ones and the bad ones, the hardworking ones and the lazy ones), pretty soon the good ones will feel that their contribution is not appreciated and they would just leave.

Answer 1

You fell for the myth that teams self-manage and now you are stuck with the unpleasant fact of most of them don't.

Yes the top 10% programmers self-manage nicely. That is part of what makes them in the top 10%. But reality is that most companies don't have programmers in the top 10 %. Most programmers like to pretend that they are rock stars who deserve special treatment but the reality is most are not.

So first your mistake is you gave away all of your authority which is really, really hard to recover from. It is always easier to start off more strict and loosen things as people produce than the other way around.

You trusted your developers to produce and they didn't produce, so how do you fix it. First step, stop trusting. Yes, they have abused your trust (generally when people think they are being ripped off by others they are, trust your gut instinct in this) and you need to rein them in.

You need to first get a handle on why the estimates are so far off. Because you have done such a poor job of managing, there are many reasons why and it will be difficult to get the information you need to fix the problem because you aren't currently asking for any information. They could routinely be promising what they can't deliver even working hard. Or they could be promising what they think you want to hear and slacking off because there is no downside to not meeting expectations. So before you can fix anything you need to know if they could meet the deadline if they wanted to or if they worked the actual hours they should be expected to work (at least 40).

I think your best bet given the ridiculous amount of latitude you gave them is to start with daily standup meetings that everyone must attend and core hours that everyone must be present (could be 11 to 3, doesn't have to be 9-6). Every day each person must stand up and say what progress they made the day before and what problems they have currently and how that might affect the deadline. You get what you expect and frankly you haven't expected anything.

So now you have to start asking questions. One of the benefits of the daily meeting is that no one wants to report they did nothing and made no progress the day before. So if the problem really is that they are playing you, just having the daily meeting should start to see more progress. When there is a problem preventing someone from making progress (like the person working on a module they need to reference is not making progress or not showing up to work until 3), then it is up to you to clear away the obstacles they have to making progress.

But it is also probably true that the time estimates are off. So you need to be looking at how you estimate tasks as well. You also have to realize that time estimation is hard and that you may need to keep records of estimates against actual time to improve estimation. Group estimates also tend to be more realistic as the over estimators tend to get shot down by the under-estimators. To get this data, you need to report time by task so you can compare it to the estimate. Make it a simple reporting system that takes no more than 5 minutes a day to fill out. And let them know it is being used toi improve the estimates for future sprints. You may also need to specifically change your time estimation form to include time for unit testing, communications, QA and fixing QA bugs, deployment, etc. MOst developers wil only consider the actual coding time when giving estimates unless the form itself forces them too.

Now because you abdicated your responsibilities as a manager, you will have some pushback to what are reasonable expectations. Well anyone who won't spend five minutes filling out a timesheet or who can't handle a daily stand up meeting (Even Agile shops do this) and core hours isn't someone you want to employ in the long run anyway. These are not unreasonable expectations. There is more to work than doing what you want when you want to do it and if you work in teams, you can't let a prima donna (who is really only an average programmer and quite replaceable) prevent you from getting your job accomplished.

Answer 2

I have a few thoughts here

1. You think you're leading, but does the team? The reason I ask is that I once was contracting for a company where my manager told me explicitly that all the developers were at the same level, but one of the developers considered himself to be the lead, when it came to one issue and one issue only--he wanted everyone to use spaces instead of tabs and put comments after the line of code instead of above it. Neither of the other developers supported this, but he felt he had the right to insist that we do this (and refused to discuss the issue). However, when leadership was actually needed he was nowhere to be found. For example, he'd asked me to take ownership of a subsystem and refused to take sides when the other developer wanted to rip out my nicely-architected system for something that was IMO less maintainable. So in this case I ignored his "leadership" on the code formatting. Lead, follow, or get out of the way, but don't only insist on leading when it's something trivial/stupid. Also, make sure that you really occupy the leadership role you think you do.
2. Are there other factors that are impacting their estimates that are beyond their control? Are your developers saddled with a load of crappy code to maintain/build on? It's impossible to estimate well under these conditions. It may be that for some reason (and let's hope it's not you), your developers are being prevented from cleaning up pain points when they're encountered. If they are prevented from fixing the underlying issues and they feel their feet are being held to the fire to live up to estimates that no one could make accurately, then they're going to be reluctant to put it in writing that this is what's happening.
3. Are you doing enough to make sure roadblocks are removed? I have found that in companies that say that finger-pointing is not acceptable, this means "don't make waves when you're not getting the resources you need," not "you won't be blamed when you can't make deadlines because someone didn't get you what you need." Stay on top of Subject Matter Experts, Designers, etc., who are further up the pipeline than your developers. When a resource doesn't arrive as scheduled, it can have a huge impact, even on developers who can switch tasks in response. Also, I have found that some designers have zero clue what I do with a file once it leaves their hands, so their work will take extra time to deal with. Many developers won't take this into account, especially if it is not socially acceptable on your team to notice or they have never worked with that designer before.
4. Are the developers, in fact, missing their deadlines? I've been in situations where the specifications were really murky, and my requests for clarification weren't met with much response. Once something was built, more requirements emerged, and these were reported as "bugs" against the features that I'd already built, not as new functionality. Additionally, everyone can be really insistent that the way the developer says is going to be the most time-consuming is the only way to build it. Once people have seen it, they decide that wasn't any good after all and want to go back and build it over some other way. Building a feature twice is building two features, and you need to see it that way.

All that said, I don't see that anyone has actually answered your question. The answer is in the repository. Look to see what people are checking in, and what issues their check-ins relate to. That can give you a lot of information about what is going on. If the designers are using a location to put assets, see what's ready and what's missing. If your build is good and you know how to build the product, build it and see where it is.

If you're sure you're good on the other points, and you really feel it's important to get everyone using your time tracking, you need to realize that you have to get everyone to stick their heads above the parapet at the same time, because no one wants to go first. This means you have to be prepared to put your foot down with some real consequences for not doing this, no matter who the "perpetrator" (or unperpetrator, as the case may be) might be. Think long and hard if you want to "go there."

One thing to consider is that if you are a bit of a "wet" leader, there may be some on your team who will be quite grateful if you provide some leadership--but you need to provide it across the board, not just on this one thing.

Answer 3

You address a lot of different issues in the question.

The Purpose

I would start with your purpose of setting the whole thing up. Is your primary goal to know how the work progresses so you have better visibility what's happening? Or maybe you want to control your team to some point. Or you treat it as a basis of reward system. (You mention all three.)

Start with answering this question as solutions to address each of them would be different.

Knowing the Work

If the main goal is to know the progress I'd strongly advise visualization as a tool that usually makes miracles. Simple tools such as task boards or kanban boards not only do help leaders to get the status of tasks being done in a team but also vastly improve knowledge about the work among the team.

What's more, they're fun. It means it usually is pretty easy to introduce them in the team. The trick is however to make people work with the board consistently. Actually, the biggest risk of working with the board is that it isn't updated. The difference between task management tool along with time tracker and task/kanban board is that in the latter case you learn about new tasks from the board so you go there anyway--there is a natural incentive to update the task whenever its status changes.

Controlling the team

You mention trust a couple of times. At the same time you point the need of controlling people. While I wouldn't say that you have to choose one over the other, I believe the least you should know which direction you're heading.

If you choose trust I wouldn't force people to account every single hour of their work every single day unless you work on time & material basis and this is a basic piece of information you need to settle accounts with your clients.

I would focus more on flow of work as a whole, e.g. measuring how quickly and how smoothly work items travel from backlog to done, and would pay attention to items that are stuck for whatever reasons. If you have a work item that is in development way longer than typically it is definitely a call to investigate the case and learn what's happening. Either you find an issue to address or, if someone lets you down, you get a clear signal about that too.

This also means that you don't enforce very strict work accounting on the team which should rather be appreciated. The more so that your team doesn't seem to be big fans of the current approach, given they fail to update data pretty frequently.

Rewards

In terms of information we're talking about here, I'd strongly discourage using it as a basis for any reward system.

First, I'm a big fan of Dan Pink's work on motivation. Actually, as you've mentioned rewards, I don't believe money motivates people in a way we used to think. Motivation is intrinsic and very individual.

What I'd advise here is learning what drives your people and trying to address these points instead of building one size fits all type of reward system, which usually introduces way more negative emotions and dysfunctional behaviors than anything positive.

It's even worse, if you base reward on the work done individually by team members you discourage helping, swarming over problems and collaborative work. If I'm 90% done I will likely to struggle with the rest alone instead of asking for help (even if it is the most reasonable and efficient thing to do) as I'm afraid it won't be "my" success, but "ours." Another perspective might be that I'm not willing to help as I have my own work to do and I'm not sure the thing you ask me about isn't sort of a swamp where I'll waste a lot of my time.

All in all, I'd discourage using data you get from task tracking systems to reward or motivate or incentivize people.

Answer 4

I'd make the observation that while you have used some Agile/Scrum terminology (Sprints, estimation) and indicate you don't use the a daily stand-up - however some aspects of what you are doing sound more command/control ("I assign tasks to the developers / I take out or put in tasks..")

If you want to have a self organising team, that (more critically) identifies and resolves their own failings (such as completing 50-60% of the work in the Sprint) then you will probably need to go further down the Agile/Sprint pathway, which means give the team more control.

Key points would be:

• the team needs to set their Sprint goals (not you)
• the team together need to estimate (not individuals)
• the team need to monitor their own performance (not you)

If you have a well-facilitated "restrospective" meeting for the team they should identify the fact they miss their targets, and more critically come up with ways to improve.

I suspect from what you have said you will find the loss of control this implies quite scary. From my side:

• I've found the retrospectives are more open and honest if I don't attend(!)
• you need to be prepared that your team may have harsh things to say about you
• you need to listen to what you team says, and act on it

If you don't like the sound of this, I'd suggest that you shoudl still discuss your issues with your team, and ask for their suggestions for improvment. If you trust them, then they will respond in a positive way.

Answer 5

A timesheet is not a way to track progress. At places where I have worked the timesheet was required to assign hours to the proper customer. It never tracked progress on the task.

Actually I take that back. One company did try to track more detailed information. We had to update an Access database through a special GUI to designate in one minute increments time spent reviewing requirements, researching, designing, coding, building, testing, debugging. We also had to account for time spend in meetings and on indirect tasks. This was so they could try to make better estimates. It never allowed them to know on a day by day basis how well we were working, or how close we were to getting the task done.

The failure to fill out a simple timecard is not related to being a developer. It only comes from understanding that it is required. If you are a contractor for the US government it is required. They can do on demand timecard checks. Most companies have now setup automated systems that send emails when they are not filled in by midnight each day.

The percentage completed is always going to problematic. Unless the tasks are in small chunks that can be done in less than a day, estimates will be off. If the task will take several days they will either have a tendency to want to commit to percentages only when forced.

You are bouncing between too little control and too much control. The daily timesheet requirements you want are very strict, but then you let them start tasks without their estimate. You are taking incompatible parts of different management methodologies and wondering why they don't mesh.

Answer 6

I have completely dropped tracking time spent on issues by teams I run. I requires no reflection from the individual and it gives no real information on the status of development. Instead, I require them to regularly update how much time is left on each issue. I find it's easier to convince people to do this as there is a clearer purpose and it's perceived as less keeping track of people and more keeping track of efforts.

I also challenge your statement that you're not a "PHB". You manage 10 developers, yet you still make individual issue assignments for all developers. That strikes me as maybe too much micro-management. I would consider trying to let developers pick their issues themselves and organize it between them. My experience is that a developer will feel more ownership over an issue if he or she picked it rather than it being assigned to them. Try to do estimations in a group planning session, rather than farming them out to individuals, as you're doing today. Again, in my experience, that will render estimates that are more accurate and with less deviation over time. It will also strengthen the sense that you are delivering as a team rather than a group of individuals on an org-chart.

Don't forget to hold retrospectives after each sprint. As you say, you often fail to achieve your goals. Explore why this is. Challenge your team to identify the reasons why you fail and suggest ways to overcome. Your team is not a black box, they need to tell you why they are failing. And then you (as a manager) need to do what is necessary to help them succeed.

Answer 7

It sounds like you're doing some Agile-ish things, but not all of it. Agile allows for flexibility in how you manage your teams, but there are things that simply must all happen in Agile, however you do them, and there's one ball that's critical to Agile which you're dropping; fast feedback.

You want the daily stand-up. Everyone gathers in a circle around a burn-down chart and tells everyone else what they got done since the last stand-up. This is a good thing for everyone; the devs get a daily sense of accomplishment, and you get daily data on what's on-schedule and what's not. If adjustments need to be made, this is where you learn that.

Now, in this standup, as a project manager, you are not an active participant. You are a "chicken". The developers, and any business analyst or QA staff you have, are "pigs". The analogy comes from a joke:

A Pig and a Chicken are walking down the road. The Chicken says, "Hey Pig, I was thinking we should open a restaurant!". Pig replies, "Hm, maybe, what would we call it?". The Chicken responds, "How about 'ham-n-eggs'?". The Pig thinks for a moment and says, "No thanks. I'd be committed, but you'd only be involved!"

The upshot is that you are "involved" in the process, but the team members are the ones who are "committed" to delivering the work. The standup is their tool to make sure they can deliver, and as such, they run it, not you. Your sole requirement is to ensure they have it and, if necessary, make sure it doesn't dissolve into a large discussion (standups should take 5-10 minutes tops). This can be a hard concept for a manager, but you seem to be into the self-managing teams paradigm, so hopefully it'll be an easier transition.

Other tips:

• Do the project estimation during a meeting of all devs. This estimation and sprint planning meeting is fundamental, and doing it corporately makes sure that all the estimates come in; the person is right there, and you should be asking for estimates of small chunks with well-defined scope; a developer should never be assigned an "epic" (large, complex project) without it first being broken down into smaller chunks. How do you eat an elephant? One bite at a time.
• It doesn't matter how much time they spend (unless they're working 12-hour days, or 3-hour days); what matters is how much they're getting done each day, and whether that puts them on track to meet their obligations. If, after working two 8 hour days, they're on pace to miss their initial estimate by half, then one of two things is happening; they bit off more than they could chew, or they're lollygagging and not getting the work done. Re-evaluate the workload for the sprint and if it still seems feasible, hold them to it.
• It is usually better to have teams each working on one project. A team should not be composed of a guy working on project A and another working on project B. Even if A and B are subprojects of the superproject C, teams pulled in different directions will eventually fracture. Teams can be as small as one person, and generally work well up to a dozen (after that it's usually good to split large teams into smaller ones).

Answer 8

In your long list of complaints I didn't see you say that they don't complete their tasks. Developers can get stuck spinning their wheels sometimes, so you do want some level of monitoring.

Monitor the team in a way that is convenient for them. There are more of them than you - so it is more important that the communication is easy for them.

How can they communicate their progress? Standups, daily reports, weekly reports, timesheets (must have a notes section for out-of-band communication) - pick the one that is most convenient for your team members

Answer 9

I see a couple problems.

The first is that it sounds like the "tasks" you have are too large. Break them down into smaller chunks - preferably ones that should be completed in a single day.

Next, don't ask them to track their time spent on tasks. You do that if the tasks are broken down by daily expectations this should be easy for you to do and the progress should be reported at a daily stand up meeting. For the most part I do agree with Amy and HLGEM about the employees self reporting this; however we need to get to that point first.

The main thing I'm after here is that if the team can't properly estimate tasks that last longer than a day then we need to train them. Best way to get there is to have them estimate small tasks. One side benefit is having people be excited about showing progress. Another is that if you have people who repeatedly can't complete easy tasks in a single day then you know who to replace. BTW, "creative" doesn't mean "when they get around to it"

Once you have them being able to complete small tasks quickly, then start moving it up to multi-day tasks. At this point they should be able to provide much better estimates that you can hold them to.

There's more but I think that's where you need to start.