What people are described as "alien-looking men," and why are they thus described?

The "alien-looking men" mentioned in the first chapter of Silas Marner are the weavers, one of whom is Silas Marner.

The opening paragraph of George Eliot's novel describes the era of the setting, one of pre-Industrial rural England, days in which there were individual weavers, wheelwrights, shoemakers, smiths of all kind, individuals who performed alone the tasks that machines soon would do, making such people anachronistic. 
In Chapter I the solitary figure of the weaver who bends under the burden of his linen or the flaxen thread creates a strange figure, indeed, as he crosses the hills, and he elicits the barks of dogs, who are frightened by his bizarre appearance.

The clever use by Eliot of the phrase "alien-looking" foreshadows the future occurrences of the narrative as Silas Marner is an isolated figure, one who rather easily incites the superstitious nature of the Puritanical villagers of Lantern Yard.