Let me start with this piece of advice from Neil Gaiman:
And I struggled with it for a long time. Still struggle with it from time to time.
I used to start a lot of projects, finished maybe tenth of them, if even that much. So I recognized it as a problem that needs to be solved. Now I finish 7-9 out of 10 I start. Not perfect, but getting there :)
The trouble for me are things with no external incentives or deadlines to finish them.
So here's how I fight those urges and "solved" it, with occasional lapses.
- time yourself and recognize the moment
- small chunks
- work every day, track your progress and push through
1) time yourself and recognize the moment
This one is experiment you do on yourself, gathering data about your base behavior. Start the brand new side project. Something you think is really cool. Work on it, mark in a notebook every day you worked on the project, even if with one sentence. Just Date, Started At: 20:17, Finished At: 21:05, Short Description (implemented login page).
Don't try to push yourself just to prove a point. Do what you would usually do on your side projects, just record it in a notebook. See how long you can keep your attention on the project. Then try to recognize a moment when it stops being fun. The moment you start to laze off. The moment you would rather do or start something else. The moment you lose interest, you feel tired and don't want to do the boring thing. Maybe you need to write 12 basic tests for your code which is boring work but will save you time later. Maybe you need to spend 2 hours to setup an environment. Whatever it is, remember it.
Remember that moment, mark it in a notebook with red ink. Keep on with your life, marking in the notebook every time you do some work on your side project. When you haven't worked on your side project for 3 days in a row, stop the experiment. You're done. If it is not finished by then, it probably won't be finished at all.
Count the days, the amount of time you worked on a project. Two weeks, a month, two months, whatever. That's your baseline. That's how long you keep your attention on the side projects. That is the amount of time you have to finish whatever personal project you start (at first. with practice you can prolong that time. But lets not rush too much). I'll be back there later, in later points.
2) small chunks
Start with a goal. Make a plan how to accomplish it before you start the next side project. Divide it into large chunks, then into smallest chunks/ tasks. Every large chunk needs to be a whole, just big enough thing that you can call finished. The large chunk can't be larger and take more time than your baseline from point 1. Those are your limits, that's how long you are interested in the projects you start. Until we push them, this is what you have to work with.
Try to recognize at which points in the project you might get bored. Note them on the plan. Write every task on a Post it note. Write what counts as finished task. Write down two things: 1 page -1 sentence every point list of things which worry and make you anxious about the project and 1 document detailing the list. Put five checkboxes by each item on the short, one sentence per point list. Check the list once a week. If the thing you worry comes to pass, mark it in the checkbox. If it doesn't that week, crisscross it.
Now you have a plan. Now you have a design document. Stick to it. No more scope creep. Any new cool feature must wait until you finish the current project. Have a separate notebook or word document where you list your cool ideas for a future feature. Then once your side project is actually finished make a new project with the goal of implementing that feature.
Start the side project.
3) work every day, track your progress and push through
Reserve 20 minutes every day for working on the new side project of your choosing, preferably at the same time every day. Forgive yourself if you miss one day but make sure that you work on it the next day. At least 5 minutes on the project. After every session, write in your notebook, with pencil on paper, what you worked on that day. Date, task, short description.
When you recognize the boring parts of the project, push through them. Make yourself do it. Force yourself to work on the boring parts for at least 5 minutes every day. Set alarm clock to 5 minutes, but work on it. What's 5 minutes? It is nothing. Everybody can work for 5 minutes.
Now you're tracking your progress in the notebook. You need to track it more. Take a ruler, a pencil and a piece of A4 or letter paper and draw a 30 day calendar. Starting tomorrow. 13th June, 14th June, etc. Up to 14th July. 5 rows, 7 columns. Draw it, mark the days. Every time you work on the side project, cross the day on the calendar you have just drawn. 5 minutes is worth the X, X marks the spot. Put finished post its into shoebox.
After the first month, look at the calendar, look at the notebook, look into your shoebox. Is the project finished? If yes, cool. If no, it doesn't matter, it will be done. Look how much we accomplished already! Look at all those Xs. Look at all those post-its in the shoebox! You should be proud, but keep on working. Draw another calendar. Can be on the same piece of paper, can be on another one. Keep marking every day you worked on your project.
And one day, the project will be done. And every day, forgive yourself if you do not work on project that day, but promise that you will work on it tomorrow, if only for 5 minutes.
And this works on everything that usually doesn't have set goals or deadlines. Physical exercise, learning new language, side projects.
With these things you are training yourself that it is okay to be bored sometimes, you are training yourself to track your progress, you are training yourself to commit yourself to the goal. You're training yourself to limit the scope to what you can accomplish in the time you have and at the same time prolonging that time you can work on a project. You are forming a habit.
Chunking things and making plans helps with number 1. (1. Feeling overwhelmed when given a project/task and thinking about not able to deliver it on time.(commitment phobic) )
Planing also helps with 2 (Unable to stay focussed on a project whose requirements keep changing) by preventing change of requirements until the project is finished.
It should be a general rule in software development but it is often ignored these days: once you finish the analysis, requirements are set into a design document and signed by business. Business says "Ok, we agree on this. This is what you do." . The requirements are not supposed to change much until the project is finished.
If they do change too much, you either scrap the project and replace it with a new one or finish the current project and follow it immediately with a new one with changed requirements.
I hope you will find this helpful.