I found this obituary about The Great Man, by J.B.Priestley in 1947 - fits many clowns:
I saw him long before he found his way to Hollywood, before 1914, when he was touring the halls in England with his juggling and trick billiard table act. He was very funny even then, and I seem to remember him balancing a number of cigar boxes and staring with horror at a peculiar box, in the middle of the pile, that wobbled strangely, as if some evil influence were at work. All his confidence, which you guessed from the first to be a desperate bluff, vanished at the sight of this one diabolical box, which began to threaten him with the nightmare of hostile and rebellious things.
And this, I fancy, was the secret of his huge and enchanting drollery though, oddly enough, it seems to have been missed - that he moved, warily in spite of a hastily assumed air of nonchalant confidence, through a world in which even inanimate objects were hostile, rebellious, menacing, never to be trusted. He had to be able to juggle with things, to be infinitely more dexterous than you or I need be, to find it possible to handle them at all. They were not, you see, his things, these commonplace objects of ours. He did not belong to this world, but had arrived from some other and easier planet.
All the truly great clowns - and Fields was undoubtedly one of them have the same transient look. They are not men of this world being funny. They are serious personages - perhaps musicians like Grock, ambassadors with attendants like the Fratellini, or hopeful inventor-promoters like Fields - who have, through some blunder on the part of a celestial Thomas Cook, landed from the other side of Arcturus on the wrong planet. They make the best of a bad business, but what is easy for us - merely picking up a bag of golf clubs or moving a chair - is horribly difficult for them. Things, that give us no trouble offer them obstacles and traps, for nothing here is on their side.