Dialogue, speaking, and conversations in a story are tricky things. Dialogue is governed by the rules of grammar and form. If you want it to be English (or at least understood), there are certain musts to be followed. Like any other rule of grammar, those concerning dialogue cannot be overlooked lest a written work be rendered unreadable -- or at least annoying to look at.
Simple as that.
The following is a sample conversation between a guy, a girl, and a talking horse about how to write dialogue in a story. That's right -- a written conversation about how to write conversations. I can't make this any easier for you, folks.
"Written dialogue -- including the punctuation -- appears inside of quotation marks," some guy explained. "Since my speech starts a sentence, it is capitalized. The non-dialogue portion is not capitalized as it is not starting the sentence," he went on.
The woman elaborated, "If a quote is the start of a sentence, it must still be capitalized -- even if it is not the start of the sentence which contains it."
"Every new speaker must start a new paragraph, too," informed the horse. "If the same speaker has multiple things to say, they may be placed within the same paragraph."
"A quoted statement will have its period replaced by a comma if it does not end the sentence which contains the quote," the man expounded. "Since this quote does end the sentence, it is terminated with a period."
"But what if a question is asked?" inquired the woman, who correctly placed a question mark inside of the quotation marks at the end of the quote instead of at the end of the entire sentence.
What if the girl had instead said, "If the quotation ends with a different punctuation mark than the containing sentence, put the sentence's appropriate punctuation at the end outside of the quotation marks"? Oh, right, the punctuation goes outside of the quotation marks with no punctuation before the final quotation mark.
"What would happen," began the talking horse, "if a quote was broken apart by explanatory words or description?"
"As long as the quote is one continuous sentence," the woman informed, "every part is set off with commas! Remember, too, that only the starts of sentences -- not quotations -- are capitalized."
"You are correct," he confirmed and ended the first sentence with a period. "Separate sentences made in the same paragraph still require their normal punctuation. Of course, if the quote is not interrupted by non-dialogue, there is no reason to break the quote. It can continue within the same quotation marks."
A statement by the man such as "I am not set off by commas" lacks any commas before or after the quotation as it is not the object of an explanatory action like he said.
"Neigh! If it is obvious who the speaker is, and they are performing no other action than speaking, it is sufficient to write only the quotation and imply both the speaker and action."
The girl stole the spotlight by doing a dance. "A quotation with an implied speaker and action may comprise an entire paragraph or only a sentence in that paragraph."
"If someone is speaking for multiple paragraphs," told the talking guy, "a paragraph that ends in a quotation must not end with a quotation mark as long as the speech continues at the start of the next paragraph.
"A quotation mark must begin the new paragraph being spoken by the previous speaker," rambled on the man. "Since I am no longer going to be speaking, I can end this paragraph's quote with a quotation mark."
The woman then took the time to explain indirect speech, in which a statement is described in summary -- not the exact words -- and therefore does not appear within quotation marks. She seems to find it useful alternative to quotes which could be considered long or dry.
"If someone wanted to quote this quote," quoth the man, "one would need to put single quotes around 'If someone wanted to quote this quote' as it already appears within double quotation marks. Someone could get crazy by quoting, 'one would need to put single quotes around "If someone wanted to quote this quote" as it already appears within double quotation marks,' which would require changing all single quotes within the quote to double quotes! Remember that single quotes are affected by this -- not apostrophes that appear in contractions and possessives."
So did you get the hang of things? Understand how quotations work now? If you don't, here are all of the writing rules for your further learning and reference, although if you structure everything you write like the above, you'll be fine.