She is an outsider because she is a woman, and the men are worried that talking to her will get them into trouble with Curley.
After a pause Crooks said, "Maybe you better go along to your own house now. We don't want no trouble."
"Well, I ain't giving you no trouble. Think I don't like to talk to somebody ever' once in a while? Think I like to stick in that house alla time?" (page 38)