How Do Email Servers Work? Please dont PARF...

[Laneco]

I will fully admit that the whole email/internet, etc., is kind of "magic" for me. I don't really know how it works, I just appreciate that it DOES - LOL!

When someone sends me an email, I may keep it for a long time depending on the subject matter. It is common for me to forward or resend an email that someone else needs who has deleted their copy. I still have my copy so I'll send it to them.

How did Clinton destroy 30,000 emails and no one anywhere has a single copy not even on a personal device? I figure that because I will sometimes save an email to my hard drive that other people probably do the same thing (I save technical type information and contract information).

I don't want to PARF this, I just want to know how this is physically possible that absolutely not one single person or one single device has one single item from 30,000 plus emails. Please remove Clinton from the discussion if it keeps it out of PARF.

thank you,

angela

[Vipergrün]

Email servers typically hold the email until deleted. If you delete the message on your personal device, in most cases it will also be deleted from the server. Then, on top of that, the storage/drive of the email server could be "wiped, and not with a rag", to make sure deleted emails cannot be un-deleted. Deleting files really means nothing. Wiping your phone before trading it in really does nothing. Data can be recovered unless steps are taken to make sure it cannot.

[wdfifteen]

Originally Posted by Laneco

I don't want to PARF this, I just want to know how this is physically possible that absolutely not one single person or one single device has one single item from 30,000 plus emails. Please remove Clinton from the discussion if it keeps it out of PARF.

thank you,

angela

It's not easy, but every instance of an email can be deleted. It's hard to do it by accident. Hitting the "delete" key just marks the space the email occupies in storage as available for future use. It doesn't remove anything. To wipe memory you have to "delete" then put something in the space. I just wiped a hard drive. It took several steps and then it took the computer about 24 ours to over write a one terrabyte disk.

[T77911S]

Originally Posted by Laneco

.

How did Clinton destroy 30,000 emails and no one anywhere has a single copy not even on a personal device?
angela

that's what I have wondered too. chances are no one wants to go against the Clintons. if she was not going to be president it might be a different story. (no, not a fan of her).

[willtel]

They used Microsoft Exchange server which is a product intended for enterprise use with 10's of thousands of mailboxes (or more) depending on the version used. Exchange stores emails in a central database and applications like Outlook or a smartphone connect to that database to manipulate messages. If messages are deleted from the central store those messages will no longer be seen from the clients any longer. Once large chunks of messages are removed from the database Exchange "re-shuffles the deck" to shrink the database and they can be very difficult to recover.

Most phones and clients don't really hold a copy of the emails that can be preserved without the original data on the database. Clients can be configured to retain offline copies but in big businesses that is discouraged because the data becomes too hard to manage.

[red-beard]

Angela,

There are copies of the e-mails, unless they were deleted as well, at the recipient (sent e-mails) and the sender if they were received e-mails.

I have multiple devices connected to an e-mail server. One device is setup to delete the e-mails after 30 days. Which means copies of the e-mails are on my PC as well as the server. The copies on my PC are kept until I manually delete them.

I use 30 days because I don't own the e-mail server (just a service) and they limit how much e-mail is allowed per account. 30 days does limit how much e-mail I can access on my mobile devices. They usually only keep a subset on the device. So the more that is left stored on the server, the more that is accessible to mobile devices.

If this is an Outlook server, it may keep everything on the server, until it is specifically deleted. At GE, this is how it was setup. Not many of us had remote access (I left GE in 2001!). At Cameron it was similar with Exchange servers (I left them in 2008).

[id10t]

There are 2 or 3 protocols involved with sending email.

The first is SMTP - this is the sending of mail, from the client (webmail or thunderbird or outlook or whatever on local machine) to the outgoing mail server, and then from that server to the destination server.

Once mail is on the destination server, a client (again, thunderbird, outlook, a webmail program, etc) accesses it via either the POP3 protocol or the IMAP protocol.

POP3 was designed around not being always connected. Connect to mail server, download all messages to local machine. Possibly delete them from the server at that point, or when deleted/removed from teh inbox on the client (this is configured client side).

IMAP was designed around being "always connected". With IMAP when the mail is checked it only downloads the header information. When a message is opened to be read, it downloads the rest of the message. Messages can be stored in folders on the server, or on the client, or both (ie at work my Inbox is on the server but any mail message over 30 days old is archived on my desktop).


I can't speak about Exchange or qmail, but both Exim and Postfix store messages on the file system, not in a database, so deletion is trivial to accomplish.

[flipper35]

I think Angela was asking why there aren't copies on the recipients devices/servers. Mrs. C used Exchange and they could have burned that unit to charcoal where nothing could be recovered but there are logs/traces/actual emails on the devices those emails were sent to in most cases. There are apps you can add into email to make the email "expire" so there wouldn't be copies on those devices, but that usually only works with encrypted mail and as far as I know the FFL did not encrypt most of those. I could be wrong, unless I am not.

[stomachmonkey]

Good question.

Been waiting for that lightbulb to go off.

Every email that is sent or received has for an indeterminate amount of time at minimum one counterpart on the other end.

We have 30k emails missing.

There was a copy of each one of them somewhere for some period of time either in an inbox or sent items.

Since the crux of the complaint is the possibility that classified information was passed through her system it's pretty much a given that were that true X number of them would exist in a Fed archive.

The Fed can not legally delete them at will and the systems are backed up.

So really all anyone needs to / needed to do was search, starting with State as it's most target rich, the archive for emails sent to or received from her private account.

Match whatever you find with what she handed over and if there are unaccounted for matches it can be assumed those might be among the 30k.

At minimum you should be able to parse the log files for the Fed servers and still find evidence of bi directional exchanges that are not accounted for by "physical" copies.

[Laneco]

OK, based on what you guys are telling me, it is possible depending on the server protocol for emails to be erased. For instance, when I was working at a large company a few years ago, the company had its own servers. Company email did indeed disappear - guaranteed. So that company server must have been similar in nature to what you are describing.

But the copies that I had loaded into my computer hard drive did not disappear nor did the backup that I store in a remote hard drive (my aww-**** contingency). No matter what happened, I had those and they proved to be most valuable on a couple of occasions...

So how did Clinton make all of those disappear for everyone who had received them and stored them either on a laptop hard drive, or backed up onto a remote hard drive? I realize that gov't employees are not supposed to back up items to remote hard drives, but it DOES get done and not everyone of those 30,000 deleted emails went to a gov't person... How the heck did she do this?

*edit - was still typing when stomach monkey wrote in - thanks

angela

[stomachmonkey]

And here's the other thing.

Spam.

This is one of my partners / clients spam filters.

It's the first line of defense in a 3 tiered system so it's not set too aggressively.

It represents about 7 months of filtering at this point.

Even with the triple filter approach the average users inbox is 1/3rd spam.

I'd say realistically only around 15% of what passes through that server is legit mail.

[id10t]

Originally Posted by Laneco

OK, based on what you guys are telling me, it is possible depending on the server protocol for emails to be erased. For instance, when I was working at a large company a few years ago, the company had its own servers. Company email did indeed disappear - guaranteed. So that company server must have been similar in nature to what you are describing.

But the copies that I had loaded into my computer hard drive did not disappear nor did the backup that I store in a remote hard drive (my aww-**** contingency). No matter what happened, I had those and they proved to be most valuable on a couple of occasions...

So how did Clinton make all of those disappear for everyone who had received them and stored them either on a laptop hard drive, or backed up onto a remote hard drive? I realize that gov't employees are not supposed to back up items to remote hard drives, but it DOES get done and not everyone of those 30,000 deleted emails went to a gov't person... How the heck did she do this?

*edit - was still typing when stomach monkey wrote in - thanks

angela

She didn't. So either the people who have something incriminating don't want to get involved, have something that can be held over them, are on her side anyway, etc. I would imagine that quite a few of them were internal to others in the state dept. who may be employed only by the political will of the incumbent office holders... Also possibly people who work for other governments, etc.

[stomachmonkey]

Originally Posted by Laneco

OK, based on what you guys are telling me, it is possible depending on the server protocol for emails to be erased. For instance, when I was working at a large company a few years ago, the company had its own servers. Company email did indeed disappear - guaranteed. So that company server must have been similar in nature to what you are describing.

But the copies that I had loaded into my computer hard drive did not disappear nor did the backup that I store in a remote hard drive (my aww-**** contingency). No matter what happened, I had those and they proved to be most valuable on a couple of occasions...

So how did Clinton make all of those disappear for everyone who had received them and stored them either on a laptop hard drive, or backed up onto a remote hard drive? I realize that gov't employees are not supposed to back up items to remote hard drives, but it DOES get done and not everyone of those 30,000 deleted emails went to a gov't person... How the heck did she do this?

*edit - was still typing when stomach monkey wrote in - thanks

angela

You've hit on my point.

There is a lot of talk about the number deleted and the assumption is that some or all of them were work related.

You don't have to find all 30K or even 5K to prove that.

You just need to find X out of 30k.

And that is absolutely doable.

It's unrealistic to think that out of a potential 30K not one of them hit a government system and there will absolutely be a record of it somewhere.

Unless they were not work related.

[VincentVega]

It would be pretty funny after all this to find 30k emails about amazon gift cards or hot russian women looking for love.

Retention policies and how its implemented are a big part of this. Rotate old email out and screw that up and poof, mail if gone. Lots of potential ways to 'lose' data. Could be as simple as a misconfiguration, something got corrupt, intentional....

[flipper35]

Originally Posted by VincentVega

It would be pretty funny after all this to find 30k emails about amazon gift cards or hot russian women looking for love.

Retention policies and how its implemented are a big part of this. Rotate old email out and screw that up and poof, mail if gone. Lots of potential ways to 'lose' data. Could be as simple as a misconfiguration, something got corrupt, intentional....

At first I was going to joke about those would be on Mr Clinton's server, then I thought more about it and seems appropriate for her as well. Not that there is anything wrong with that.

[flipper35]

Oh, as far as misconfiguration, her people can only control her equipment. Unless the NSA was involved, then they might be able to scrub a lot of systems.

[KFC911]

I haven't read most of this thread, but I'll add this...I've been out of the large corporate game for eight years now, but EVERY single email I ever sent or received could be retrieved if necessary going back over 20 years. NOTHING was ever deleted without a permanent backup on tape. Email was just the tip of the iceberg....I could also data mine VPN, firewall logs etc. and tell the legal dept. exactly what a user was accessing too...I didn't have to worry about big brother watching me...I was he

[VincentVega]

I've seen a few cases where something was supposed to be quarantined yet the target disk was full and things got hosed. I've also seen a backup routing fail leaving holes in you cant explain.

No parf intended, but this type of thing can be expected in a home brew system. Or, someone can rm * and everything is out the window.

[stomachmonkey]

I've seen some bizarre stuff.

This one goes really far back to the dial up days before companies started dropping T1's.

Our Japanese office was the absolute worst at communicating. Seemed like sometimes it took days for them to reply to things.

Our CEO used to get really hot under the collar when he'd send something to the President of our APAC office and not hear back till the next day.

He'd call to chew the guy out only to hear the excuse that he had replied right away even though his email was no where to be found.

I'm sitting in the VP of IT's office one day when the CEO calls and proceeds to give him a particularly unpleasant reaming when it dawned on me.

CEO hangs up and I look at VP and asked, how does the mail server deal with mail from the future?

He looked at me like I was nuts then went "ohhhh"

Hint, it was holding inbound mail till the emails "future" timestamp matched the local time on the receiving end.

[mistertate]

My employer had an outlook email server crash this year and most of us lost everything. I assume my customers lost the same emails I lost. It was a PITA and embarrassing.