Lack of a doctype triggers quirks mode, which is meant only for backwards compatibility for "legacy code" that was created before people started using doctypes. It should pretty much never be used; you should always declare a doctype.
Which one to choose?
In this day and age, this is all you need:
<!DOCTYPE html>
You can continue to use XHTML syntax with this doctype if you wish. As far as CSS goes, there aren't any differences I'm aware of with different doctypes, as long as you have one. Doctypes will however change which attributes and elements are valid and in which context. Use the W3C Validator to test your HTML.
Unfortunately, this means you will be rewriting much of your CSS to work in standards mode. I know it sounds like a chore, but you'll just have to bite the bullet and rewrite it.
Important note for moving forward: remove the inline CSS and use an external stylesheet instead, otherwise (among other things) you will find maintenance to be a total nightmare.
Of interest: http://hsivonen.iki.fi/doctype/#choosing
Choosing a Doctype
text/html
In a nutshell: Here are simple guidelines for choosing a doctype for a
new text/html document:
Standards mode, cutting edge validation
<!DOCTYPE html>
This is the right thing to do unless you have a specific reason to avoid it. With this doctype, you can validate new features such as
<video>, <canvas> and ARIA. Please be sure to test your page in the
latest versions of the top browsers. Standards mode, legacy validation
target
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
This doctype also triggers the standards mode, but lets you stick to legacy validation in case you want to avoid new features or more
precise validation of old features. You’d like to use the Standards
mode, but you use sliced images in table layouts and don’t want to fix
them
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
This gives you the Almost Standards mode. Please note that your layouts based on sliced images in tables are likely to break if you
later move to HTML5 (and, hence, the full Standards mode). You
willfully want the Quirks mode
No doctype.
Please don’t do this. Willfully designing for the Quirks mode will come and haunt you, your coworkers or your successors in the
future—when no one even cares about Windows IE 6 anymore (already no
one cares about Netscape 4.x and IE 5). Designing for the Quirks mode
is a bad idea. Trust me.
If you still want to support Windows IE 6, it is better to apply specific hacks for it using conditional comments than to regress other
browsers into the Quirks mode.
I am not recommending any of the XHTML doctypes, because serving XHTML
as text/html is considered harmful. If you choose to use an XHTML
doctype anyway, please note that the XML declaration makes IE 6 (but
not IE 7!) trigger the Quirks mode.
