Why does it take Walter Cunningham so long to make a decision concerning Jem's suggestion?

Scout is mad at Walter Cunningham. She blames him because she got in trouble at school with her new teacher over a discussion about his lack of lunch. She fights Walter until Jem comes up and tells her not to. Walter seems worried that both Jem and Scout will beat him up. Scout mentions to Jem that Walter does not have lunch. Jem decides to invite Walter over to their house for lunch. At first, Walter seems excited about this, but then his face changes. He seems concerned about going over to their house for lunch. He fears being beaten up. He is also concerned about being around Scout, who is angry with him. To reassure him, Jem tells Walter that Atticus knows his father. He also promises Scout will not try to beat him up anymore. Scout does not want to make any promises about not beating Walter up, but she eventually relents. She tells Walter about the delicious butterbeans Calpurnia cooks. Jem and Scout leave for home, and Walter eventually follows:

Walter stood where he was, biting his lip. Jem and I gave up, and we were nearly to the Radley Place when Walter called, "Hey, I'm comin'!" (Chapter 3)