"Why did they depart in the beginning?"
There is probably more than a single answer to this question. It began in 29.11.1947 when the UN decided to split the land to two states. The Arabs rejected that decision, and war began. Arabs were fighting Jews and Jews were fighting back for about 2 years.
1. During that war, some Arab nations promised the Israeli Arabs that they come to rescue them, so some Arabs left their homes in order to stay away from the battlefield until the war is over.
2. There were probably some Arabs that were deported by Jews during the war, because they were shooting nearby Jewish settlements.
3. And, there were maybe some innocent Arabs that were deported by Jews during the war.
of all three groups, the only one that MAYBE has a right to return is group 3.
Speaking of the "right of return", it is insightful to read the covenant of Hamas, who were chosen by the Palestinians in a democratic elections to be their leaders:
Article Eleven: "This is the law governing the land of Palestine in the Islamic Sharia (law) and the same goes for any land the Moslems have conquered by force, because during the times of (Islamic) conquests, the Moslems consecrated these lands to Moslem generations till the Day of Judgement." (see http://www.yale.edu/lawweb/avalon/mideast/hamas.htm )
So, according to the Palestinians, Muslims ALWAYS have a right to return to ANY land that have ever been Muslim, while other religions NEVER have a right to return to a land conquered by the Muslims.
If that's what they believe in, I don't think they can claim any "right to return".