Yes, often overlooked. And, I hear, almost impossible to selfhost these days without a degree in CS, because “we block all non big tech e-mail providers”.
I have a degree in CS… actually spent some time implementing email protocol as part of a class to send test messages through I think websockets in Java or something. It was really interesting and kind of a cool project.
Yeah, I ain’t touching that shit. I’ll more than happily let my domain name provider manage that for me so I can focus on bigger and better things going through yet another Civilization 5 Vox Populi campaign.
Yea and no. The email “big tech club” happened under a pretense of spam blocking because back then spam and bots were a new concept. We now know a bit more about it and have anti-spam measures built into, e.g. even lemmy so I don’t think big tech will be able to use spam itself to piggy back on. At the same time Facebook has already announced Fediverse integration, and while there’s a petition to defederate from it as soon as they bring their servers up - what’s going to happen if Facebook+Twitter+Reddit decide to hop on the fediverse bandwagon? There’s just too much juicy content there right now. The FOSS Fediverse will have a tough time choosing between accessing all of that juicy content or keep the team values up. Now all of the mentioned sites are in decline, so as long as FOSS fediverse gains momentum faster than Big Tech unites I think we’re safe.
However, I’m not so sure we’d be able to avoid it, even if big tech did not get involved.
For example, beehaw.org does not federate with a lot of other instances. How long until a few big lemmy instances decide they are important enough to block every new instance by default? Probably not going to happen anytime soon, but it could be a foreshadowing.
I think people should be incentivised to join the smaller instances. But I guess most new people coming to the fediverse don’t know that you can talk with other instances.
Well critical mass is the issue here. Social media sites are “fun” only if there are people there posting, talking. So it’s perfectly fine is there are say, 5 fediverse isolated islands that don’t talk to each other as long as each island has critical mass it‘ll be sustainable.
Also the benefit of fediverse is that I don’t need to understand that different communities are on different instances - I just go into search and find communities I want. Yeah some will have funky @names appended to them but I don’t need to understand what that means in order to interact with those communities.
I’ve heard a lot of people say it, but it feels like FUD - it really isn’t as hard as people say. I host my own mailserver (well, I use a VPS, so the provider does the hardware and the networking, but I do the mailserver). And I don’t have a degree in Computer Science (well, I do have a PhD in Bioengineering that involved software-adjacent research, and did a few undergraduate CS papers - but none of that covered anything relevant to mail, so is not relevant here).
Delivering from commercial rather than residential IP space probably helps - and try to get an IP address that hasn’t been previously used by a spammer. If you set up DMARC with DKIM and SPF (lots of abbreviations, but look them up if you want to set up a mail server), you will be deliverable (at low email volumes) to the majority of places already. Microsoft is the one exception, but just send a bit of email to them that isn’t spam (can be to one of your own accounts), and if need be fill in their form to complain you are being blocked incorrectly, and you will get deliverability at low volumes.
All of that is enough for normal personal or business emails at normal volumes to be deliverable pretty much anywhere you like, and it doesn’t take long to get there. If you want to send legitimate bulk newsletters and the like, it is a bit harder (basically, you have to warm up the IP and not grow too fast), but also not impossible - but understandable businesses doing that want to pay for someone else to have done that for them, and that is where advice not to run your own server comes from. But for the average person who doesn’t run mailing lists, getting deliverability is not that hard. And the more people who run their own servers, the harder it is for those who would make email a walled garden to get their way on it - so it is for the greater good.
The bigger challenge with running your own server is actually not other’s anti-abuse measures, it’s that you have to have your own anti-abuse measures if you don’t want a mountain of spam and your logs filled with people trying to brute force you. It isn’t that hard, but a few tips: 1) install packages for your mailserver from your distro, and update early, update often, 2) use keys rather than passwords for protocols (like ssh) where you can, and use long secure passwords where you have to (e.g. for IMAP and submission logins), 3) read the docs for your mail server to make sure you aren’t set up to be an open mail relay please! Check authentication is required with a secure password or you will get blocked by everyone as a spam source, 4) use something like fail2ban to block brute forcers before they fill up your logs, 5) consider SpamAssassin and postgrey (or some other greylisting solution) to cut back on spam, 6) there is a tiny handful of ASNs on the Internet that allow spammers and don’t take any meaningful action against them. You can do a daily download from https://archive.routeviews.org/oix-route-views/oix-full-snapshot-latest.dat.bz2 and bzgrep it with a command like bzgrep -e " (213035|400377|399471|210654|46573|211252|62904) i"$TEMPDIR/snapshot.bz2 | cut -d" " -f 3 | sort | uniq to get a list of IP ranges that are more likely to be spam than anything else (obviously, never use one of those providers since they are widely blocked). Script blocking traffic from them with iptables, and your spam volume will fall greatly. 7) if a spam does get through, check the headers and report it to the ISP it came from - you will be helping to shut down spam for everyone, making it easier to run small mail servers.
I run my own mail server for US$5/month (on VPS compute resource shared with other things, not just for mail), it is mostly automated (and can be redeployed with Ansible if need be), I rarely need to touch it aside from checking upgrades are working etc…, and I haven’t had a deliverability problem in ages, and I now get pretty minimal spam (and the spam that does get through is reliably filtered by Thunderbird’s spam filters at the client).
So please don’t let the naysayers deter you from self-hosting your mail, it’s really not that hard, especially compared to the constant stress that providers like Google might lock you out of your account for vague “security” reasons, shadowban you, sell information from your emails for marketing, decide to cancel the service, or whatever other abusive thing they dream up next.
Thanks for the reply and the technical details. And great to hear that hosting your own mail is still possible for us mere mortals. That’s the world I want to live in!
Yes, often overlooked. And, I hear, almost impossible to selfhost these days without a degree in CS, because “we block all non big tech e-mail providers”.
Probably even with a CS degree.
It’s just a hassle to maintain, and too mission critical to have it go down.
I wonder if the same won’t happen with the fediverse, if we let some instances get too large.
I have a degree in CS… actually spent some time implementing email protocol as part of a class to send test messages through I think websockets in Java or something. It was really interesting and kind of a cool project.
Yeah, I ain’t touching that shit. I’ll more than happily let my domain name provider manage that for me so I can focus on
bigger and better thingsgoing through yet another Civilization 5 Vox Populi campaign.Going down isn’t the problem. Keeping an email server alive isn’t difficult.
Your messages getting summarily rejected by just about everyone is the problem.
Yea and no. The email “big tech club” happened under a pretense of spam blocking because back then spam and bots were a new concept. We now know a bit more about it and have anti-spam measures built into, e.g. even lemmy so I don’t think big tech will be able to use spam itself to piggy back on. At the same time Facebook has already announced Fediverse integration, and while there’s a petition to defederate from it as soon as they bring their servers up - what’s going to happen if Facebook+Twitter+Reddit decide to hop on the fediverse bandwagon? There’s just too much juicy content there right now. The FOSS Fediverse will have a tough time choosing between accessing all of that juicy content or keep the team values up. Now all of the mentioned sites are in decline, so as long as FOSS fediverse gains momentum faster than Big Tech unites I think we’re safe.
The Facebook bit is my concern as well.
However, I’m not so sure we’d be able to avoid it, even if big tech did not get involved.
For example, beehaw.org does not federate with a lot of other instances. How long until a few big lemmy instances decide they are important enough to block every new instance by default? Probably not going to happen anytime soon, but it could be a foreshadowing.
I think people should be incentivised to join the smaller instances. But I guess most new people coming to the fediverse don’t know that you can talk with other instances.
Well critical mass is the issue here. Social media sites are “fun” only if there are people there posting, talking. So it’s perfectly fine is there are say, 5 fediverse isolated islands that don’t talk to each other as long as each island has critical mass it‘ll be sustainable.
Also the benefit of fediverse is that I don’t need to understand that different communities are on different instances - I just go into search and find communities I want. Yeah some will have funky @names appended to them but I don’t need to understand what that means in order to interact with those communities.
I’ve heard a lot of people say it, but it feels like FUD - it really isn’t as hard as people say. I host my own mailserver (well, I use a VPS, so the provider does the hardware and the networking, but I do the mailserver). And I don’t have a degree in Computer Science (well, I do have a PhD in Bioengineering that involved software-adjacent research, and did a few undergraduate CS papers - but none of that covered anything relevant to mail, so is not relevant here).
Delivering from commercial rather than residential IP space probably helps - and try to get an IP address that hasn’t been previously used by a spammer. If you set up DMARC with DKIM and SPF (lots of abbreviations, but look them up if you want to set up a mail server), you will be deliverable (at low email volumes) to the majority of places already. Microsoft is the one exception, but just send a bit of email to them that isn’t spam (can be to one of your own accounts), and if need be fill in their form to complain you are being blocked incorrectly, and you will get deliverability at low volumes.
All of that is enough for normal personal or business emails at normal volumes to be deliverable pretty much anywhere you like, and it doesn’t take long to get there. If you want to send legitimate bulk newsletters and the like, it is a bit harder (basically, you have to warm up the IP and not grow too fast), but also not impossible - but understandable businesses doing that want to pay for someone else to have done that for them, and that is where advice not to run your own server comes from. But for the average person who doesn’t run mailing lists, getting deliverability is not that hard. And the more people who run their own servers, the harder it is for those who would make email a walled garden to get their way on it - so it is for the greater good.
The bigger challenge with running your own server is actually not other’s anti-abuse measures, it’s that you have to have your own anti-abuse measures if you don’t want a mountain of spam and your logs filled with people trying to brute force you. It isn’t that hard, but a few tips: 1) install packages for your mailserver from your distro, and update early, update often, 2) use keys rather than passwords for protocols (like ssh) where you can, and use long secure passwords where you have to (e.g. for IMAP and submission logins), 3) read the docs for your mail server to make sure you aren’t set up to be an open mail relay please! Check authentication is required with a secure password or you will get blocked by everyone as a spam source, 4) use something like fail2ban to block brute forcers before they fill up your logs, 5) consider SpamAssassin and postgrey (or some other greylisting solution) to cut back on spam, 6) there is a tiny handful of ASNs on the Internet that allow spammers and don’t take any meaningful action against them. You can do a daily download from https://archive.routeviews.org/oix-route-views/oix-full-snapshot-latest.dat.bz2 and bzgrep it with a command like
bzgrep -e " (213035|400377|399471|210654|46573|211252|62904) i" $TEMPDIR/snapshot.bz2 | cut -d" " -f 3 | sort | uniqto get a list of IP ranges that are more likely to be spam than anything else (obviously, never use one of those providers since they are widely blocked). Script blocking traffic from them with iptables, and your spam volume will fall greatly. 7) if a spam does get through, check the headers and report it to the ISP it came from - you will be helping to shut down spam for everyone, making it easier to run small mail servers.I run my own mail server for US$5/month (on VPS compute resource shared with other things, not just for mail), it is mostly automated (and can be redeployed with Ansible if need be), I rarely need to touch it aside from checking upgrades are working etc…, and I haven’t had a deliverability problem in ages, and I now get pretty minimal spam (and the spam that does get through is reliably filtered by Thunderbird’s spam filters at the client).
So please don’t let the naysayers deter you from self-hosting your mail, it’s really not that hard, especially compared to the constant stress that providers like Google might lock you out of your account for vague “security” reasons, shadowban you, sell information from your emails for marketing, decide to cancel the service, or whatever other abusive thing they dream up next.
Thanks for the reply and the technical details. And great to hear that hosting your own mail is still possible for us mere mortals. That’s the world I want to live in!