Answer: Your feelings of guilt are guiding you towards the ethical choice. Hiring and training someone is costly, so your employer deserves a chance to recoup that investment. If you ditch this job now, you will be treating your new boss just like the old one treated you.
This is also a practical decision, because two very brief jobs will not look good on your resume. Try to stay with this employer for at least a year and use that time to carefully study the job market and define your career goals. You need to be sure that your next position is a keeper.
If you are pondering a job change, see Five Steps to Finding the Right Job.