[discussion] writing the perfect php mail() smtp system

[atesin]

hi... new user here...  i am very glad to be here

imagine a story...  you are a volunteer of a non-profit organization that helps people in africa (animal conservation, find water, child healthcare or some like) ... but your talent is to build computers and run servers, so you offer them to develop a website to spread the cause

so, as they have no money (and you either) you go to some junkyards and build a pc from scraps, and install linux, and http server, and colocate the server in your house, connect to internet using your home connection, configure router for outside reachability (by port forwarding, dmz or whatever), install dynamic dns...  and put a website with information about your work and some contact info

soon you realize that a better way to being contacted (besides publishing your oranization phone number, email address or physical address) is to put a contact form page, faster and more comfortable for who wants contact to offer/search for help... you contact a couple people a week, but is very important for everyone

but mail servers costs money and your people dont have it (like you), they are poor farmers so they have no even banks... so you search and find some free email services that can also send email by smtp (gmail, hotmail, yahoo, yandex, etc), so you pick and configure one of them.... this is when your problems start

you quickly learn that to make the system work is not so easy because of the SPAMMERS... they previously paved your road full of obstacles, so now everyone are paranoid about spam, and now mail delivery systems are very very restrictive... so you have to think in develop something anti-spam system

first you have to design some bulletproof contact form with many anti-spam protection measures like captchas, unique hidden codes, referer checkings, short live tokens, time synchronization, timeouts, ip/cookies lookups, browsing sessions, etc etc etc.. so your PUBLICLY ACCESIBLE contact form wont be found and exploited by spammers with bots or anyway

next step is make php mail() function work in php, next step is sendmail (or similar) work in linux, next step is to make your mta (outgoing mail program) and your external mail delivery service talk each other, next step is ensure you wont be blocked by your external service

.....  i read that heliohost also offer mail delivery service, but is also put in blacklists because of some spammers (ex?) members...  so i think maybe the best option is to rely in some external mail delivery service using smtp, and use php mail() function to give them the mails to be sent...  but mail() function is also blocked here due bad experiences with (fcking) spammers...  so i was thinking, what way could we enable and make use of mail() function safely?

certainly in my website (currently hosted on my house, searching a free and more reliable webhosting... my isp scratches his back and my websites go to hell) i use mail() function sometimes to send important messages like account creation validation or some desperate people looking for help (our real website is more related to women and alcoholism) and i am interested in make it work

by give users mail() function access BUT forcing them using an external smtp server, heliohost frees the responsibility and risk of being blacklisted (anyway the system wont work other way because are already blacklisted)...  so..  how this could be done??, i started by doing some research to write a /bin/sendmail wrapper

i was thinking many ideas to be implemented on my home server, one of them are sending quotas in a certain amount of time.. say 12 mails/hour, that is not the same as 12 mails each clock hour... think on a database that take note the time each time you send a mail, and calculate the time difference between this mail and the 12th last sent email wont be more than 1 hour, so this way you can safely send an email each 5min, or if you prefer send all 12 mails at once having to wait 1 hour STARTING FROM NOW to send mails again

another system i actually implemented in my server is a /bin/sendmail wrapper that accepts an additional parameter, so if some hacker/spammer accesed my filesystem and manages to inject some script with mail() function, it wont work because he doesnt know the additional parameter code.... before this i had to do some tests/research to make it work to know EXACTLY how mail() function and /bin/sendmail works together internally

going further, i imagine a send mail code generator in admin panel, that accepts the path where the mail sending script (the one containing the mail() function) is located, that generates a unique additional parameter code depending the location of the script, that you have to put in your actual script code... maybe this could be done and would be fantastic

but if mail() function doesnt/wont work, maybe this could be made work with some external php mail library like PHPmailer or SwiftMailer... i never tried before (remember you cant send email directly and have to use an external smtp server, because of heliohost ip blacklisting) ...  i found this related page = https://forum.infinityfree.net/t/how-to-send-email-with-gmail-smtp/49239

 

 

[Kairion]

Hi, atesin.

You are starting from a false premise that the PHP mail() function is not enabled at HelioHost servers, but that is not the case, as this function is enabled, and any user is free to use it. If users do not want to use that function, they can also use external PHP mailer libraries, such as PHPMailer, which you pointed out.

About being listed on blacklists, it happens with every single email provider. When it happens, the provider must deal with that. In HelioHost's case, we count on our users to let us know if they have any issues with their emails, and we fix them when that happens.

We also do have the following email quota limits (which can be changed upon request to our root admins, which may or may not allow it), which help us to stay out of blacklists:

• Account's email quota of fifty (50) emails per day,
• Email account's quota of ten (10) emails per hour.

After someone reaches any of those limits, that someone is unable to send new emails until the limit is reset at midnight UTC.

You need to note that any solution you adopt is subject to being blacklisted/undelivered, as it is outside of the sending server/provider scope (even if the sending server/provider is not on a blacklist, the email can still not be delivered or be flagged as spam, as it is solely a decision made by the receiving email server), meaning there is absolutely no 100% way of guaranteeing your emails will be delivered to someone else's email server, much less to their inboxes.

If you want to have a different email provider, there are some free alternatives:

• Zoho Mail has a "free forever plan", but unfortunately it does not allow IMAP/POP3/SMTP connections (email access is only possible through their mobile apps or via their webmail on the free plan) but could be useful if a user only needs free email hosting (up to 5 free email accounts).
• For sending bulk emails, you can check https://influencermarketinghub.com/free-bulk-email-marketing-services/, which has a list of some services that may provide free sending bulk emails, though with some limitations or on trial plans.

[atesin]

hi....  i dont remember where i read mail() was disabled, but thanks for clarify

is there some documentation on how can we configure mail() / sendmail here in heliohost?  ..  for example, to use an external smtp server, with different ports and auth protocols (tls, starttls, xoauth2, etc), different hostnames, etc?

i am doing some tests with PHPmailer... seem very powerful (no need to try it before) ...  i am finding how to write my own php mail() function on top of it, did some test and luckly tcp ports i tested are open.... i can tell you what i installed in my server...

i have no postfix or other mail server installed .. i never tried because i found configuring a mail server a titanic task, i always skip and never dare to try (need static ip by isp, configure dns, configure mailboxes, configure rules, configure certificates, configure filters, configure routes, and who knows what more... )...  i had seen people configuring postfix/exchange before and looks a real pain in the assss ... 

....  so i took advantage on mail servers already established, with mailbox, certificates and everything configured and woking, and just connect by smtp

so i installed ssmtp or msmtp clients on my server (the latter also works in windows), and use as sendmail bin ....  i wrote a bash wrapper to enable multiple mail accounts/services/configurations, and to add an extra layer of security...  then set php sendmail_path variable to it ..  and in mail() function i need to add a third parameter according the desired mail configuration

these days i noticed the php variable mail.add_x_header to add the mail() caller php script path.... with a wrapper we can intercept this header to get the originating script, and generate a unique token for that script in the admin panel, that must be added as additional mail() parameter

i am a little crazy

[Kairion]

On 3/6/2023 at 1:44 AM, atesin said:

is there some documentation on how can we configure mail() / sendmail here in heliohost?  ..  for example, to use an external smtp server, with different ports and auth protocols (tls, starttls, xoauth2, etc), different hostnames, etc?

By default, the PHP mail() function can only send emails through your local mail server and cannot be configured to use external SMTP servers. However, if you are on a Virtual Private Server (VPS), you can change your Mail Transfer Agent (MTA) settings, such as sendmail (as you mentioned), to use an SMTP relay. This allows your mail() function to send messages through your local server, which then uses the external SMTP server to send the emails.

For example, I have set up my MTA to relay messages through Google Workspace's server. This enables me to use Google Workspace's security features and also keep copies of the emails my server is sending.

It's important to note that this configuration affects all users on the same server, so it's not possible on shared hosting environments like Tommy, Johnny, Ricky, and Lilly.

If you want to use external SMTP servers on a shared host, you will need to use a library such as PHPmailer or Symfony Mailer or create your own.

Regarding your comments about your customizations: While customizing software can be useful for learning and testing purposes, it's important to be cautious when making changes to standard software in a production environment. Any customizations should be thoroughly tested, and you should be aware that any changes to standard software may require you to fork and maintain the customized version with its upstream version to avoid security risks and ensure compatibility, which usually is more trouble than it is worth.

[atesin]

hi kairion, i appreciate your interest in the conversation... i am always passionate to learn new things

do you know msmtp?

i know original sendmail is an ancient program (maybe from 70-80's) and was possibly the only (or the easiest) way local users were able to "send mail" that days (hence the name) (but you need an outgoing mail service running locally).... so as it was a popular program, many later applications have "sendmail mode" like qmail or postfix (i never installed, i always try to skip the headache xD).... but ssmtp and msmtp (and others?) are special as they are small outgoing mail CLIENTS compatible with "sendmail mode"

i mean, to send mail locally, normally we have to install and configure an outgoing mail server on your machine like postfix (a real headache)... to operate in "sendmail mode" like others, postfix have a /bin/sendmail binary that is overwritten when installed... so normally you have to install postfix, then configure the mailbox, dns, aliases, queues, network, etc etc, so you can then use /bin/sendmail [sendmail options] to send mail same way than before... posfix sends the mail directly to recipient mail server through smtp, the protocol that mail servers use between them

but instead install and configure a whole mail server yourself, you can get a mail account in some already working mail server, and just install a "sendmail mode" smtp client to connect that server and let send your mails =D .... with some "sendmail mode" smtp relay (like ssmtp or msmtp) you can actually use  php mail() with an external mail server  ...  configuration is tricky but easy, can even make it scriptable so that opens a lot of possibilities, i have also done in windows because windows php mail() sucks  ...  the only little downside is your mail address will have the "@extserver" suffix, but i dont care considering i didn't have to install the server and is not consuming MY disk space =D

i mean... imagine msmtp is installed server wide, and an admin panel is added to configure parameters by user.... take a look of ssmtp and msmtp:

• https://wiki.debian.org/sSMTP
• https://marlam.de/msmtp/

 

[Kairion]

Hi @atesin,

I do am aware there are such alternatives depending on someone's using scenarios, they just do not fit the HelioHost's use-case, as someone hosting here will:

• Like to have an actual email server (i.e. sending and receiving emails through their (sub)domain),
• Not use email at all,
• Use a script/software/library that already has SMTP support for sending emails through an external server.

Changing from Postfix to any of those alternatives would remove a hosting feature and break the use for an uncountable number of our users.

Using any of those alternatives you have pointed out is possible on a VPS, where you have full control of the entire operational system and it is all yours, thus you can install and manage it however you like, without the need to take into consideration anyone else's use-case.

[atesin]

oh i see ... "we already have postfix"

i didnt know that we have postfix, and why i didnt imagine? xD ... i understand having posfix is better than to have some smaller smtp relay,,, and that the most common user case is to want a full featured smtp server to use email with his/her own domain, something not possible with a simple smtp relay

i also understood that maybe i am asking for some elaborated solution, and that i am not in my home server anymore (i feel dumb)... so i think the better alternative i have to not to bother others is what you suggested me: to try a php mail library.....  i am already doing tests with PHPmailer, is a little hard to adapt my scripts, but i am learning

thanks for your patience