Sometimes back I had talked about how to make sure the mails can be deleted from your mail server also when you delete from Microsoft Outlook. The answer was to set the purge option.
However I was looking back to the settings in Gmail with IMAP ( Which you can use it with Outlook or any other mail client ) they had give exclusive options to what happens when a mail gets deleted from the Outlook.
So the settings here will override your options in the mail client. You can set here to delete forever or just move to trash ( a better option ) when you delete any mail from your email client. You will find these under settings.
