I will repeat again, you can not mess up your character when selecting talents. You can always see a Rogue trainer and restart your talent choices. Feel free to experiment to what best fits your play-style and worst case, it costs you a few aion gold to change your mind.
Assassination Tree - This tree deals mainly with increased damage from your rogue skills. It also has a few talents that work with talent points. If you plan on using daggers, this is more than likely the talent tree where you will spend the majority of your points.
Combat Tree - The Combat tree deals mostly with weapon skills. It allows you to hit more often or gain a certain benefit to using a certain weapon. It increases some defensive stats as well. Specializing in this tree means you are looking at increasing your chances of doing damage with a weapon and your auto-attack skill, not any actually rogue skill with a couple exceptions.
Subtlety Tree - The Subtlety tree deals with exactly that, skills that are from stealth mode. This is definitely the most rogue-ish of all the talent trees. It can increase you stealth rating, your speed in stealth, the return to stealth from Sap as well as the damage and energy costs of certain skills available in stealth mode. Master of Deception is a talent that nearly every rogue will have as it increases your stealth rank by 3, a GREAT asset.