Stress on the “all of us” –it is not y/our freedom that is envied.
Well, except that Fanon says explicitly, and Sartre echoes, that the colonial act functions to dehumanize the “native,” and that the European becomes a man through that act of dehumanization. So the “us” that is “all of us” is all of us men, all of us Europeans.
So sure, many of “us” find ways to turn profit into subversion, but I think Sartre’s point is that we still don’t get to opt out of the “us,” or to disingenuously claim that we haven’t profited. We only get the agency whereby we can perform the subversion, even if utterly unintentionally, at someone else’s expense.