The DMPPcan be installed on-premise Azure, AWS, and Google Cloud or any responsibly managed data centre with proper service and access control.
However, installing the DMPPĀ requires client-side knowledge of Linux administration. It is only recommended if the client already has Linux running on-site and is comfortable with setting up and administering Linux servers.
The DMPPĀ runs best on bare metal, and since the servers are quite large and containerised - it is not necessarily a good idea to add further virtualization.Ā Ā We recommend the following three stacks.
Bare Metal + Ubuntu / Redhat + Docker
Bare Metal + VMware + Ubuntu / Redhat + Docker
Cloud + Ubuntu /RedHat + Docker
D&MCS does not support Hyper-V.
The client is responsible for server OS updates and patching. D&M is responsible for updating and maintaining the DMPP application.Ā
Machine and OS for Custom Installations
The DMPPĀ can either be installed on a single machine or in a cluster. If you want to scan more than 25TBĀ of data or more than 2000 users, we recommend setting up a cluster. As a general rule the storage size of the custom installation should be 20% of the size of the data that is going to be indexed.
Type | Trial /PoC | Single ProductionĀ Server |
OS | Ubuntu 24.04 | Ubuntu 24.04 |
RedHat EL 8.8 | RedHat EL 8.8 | |
RAM | 128 GB | 160GB |
Threads | 16Ā | 36 |
Data Disk* | 2 TB NVMe | 4-8 TB NVMe |
Size | < 20Ā accounts andĀ < 1 TB data | < 2000Ā accounts and < 25 TB data |
Ā Ā Ā ClusterĀ | |||||
App server1 | Node 1 | Node 2 | Node 3 | Node 4 | Node 5 |
Ubuntu 24.04 | Ubuntu 24.04 | Ubuntu 24.04 | Ubuntu 24.04 | Ubuntu 24.04 | Ubuntu 24.04 |
RedHat | RedHat | RedHat | RedHat | RedHat | RedHat |
256GB | 64GB | 64GB | 64GB | 64GB | 64GB |
36 | 12 | 12 | 12 | 12 | 12 |
2 TB NVME | 4% of total data sizeĀ | 4% of total data sizeĀ | 4% of total data sizeĀ | 4% of total data sizeĀ | 4% of total data size |
There are several factors that influence how much data a cluster can handle, as a rule of thumb 150 million data sets per cluster. For each additional node the cluster can handle 20-30 million additional data sets.Ā | |||||
* Disk should be formatted using ext4 /Ā Raid 1
**For single servers, the storage should be 20% of the target storage and no less than 4TB
for Cluster, each of the nodes should combine to have 20% of the total data size that is going to be indexed, evenly divided over all the nodes. In total, a 7-server setup is a minimum. Additional nodes can be added over time to scale. All nodes must be in the same subnetwork as the primary toolbox.
High-level infrastructure
Network & DMZ
The DMPPĀ must be placed in a DMZ / subnetwork.Ā
Please don't enable any local firewall on the Ubuntu / Redhat servers. All data traffic controls must be handled on the DMZ firewall network level.
The only incoming access to the subnetwork should be from companies internal network on 443 (80 redirects to 443 but must be reachable to facilitate automatic certificate updateĀ )
The DMPPĀ need outgoing internet access on port 80 and 443 for installation and monitoring.
The following ports must be opened on the DMPP, and ports marked as internal should not be accessible from outside the subnetwork.
PortĀ | Protocol | Usage |
22 (Internal use only) | SSH | Installation and maintenance | Must allwasy be avalable for D&M DevOps |
80 (Incoming) | HTTP | Redirects to 443 - must be reachable for automatic updates of certificates |
443 (out) | HTTPS | To access the toolbox admin interface and for the end-user to access reports.Must be accessible to customer users and Data & More support team from the internal network. Can be IP restrain for Incoming traffic. NOT for Outgoing. |
During installation.
Be aware that during installation, the toolbox will access several domains to get patches and updates, including but not only the domains listed below. So, please don't restrict outgoing internet access during the installation. The list below is not a af complete list of domains, as this providesĀ subdomains and load balancersĀ
*.ubuntu.comubuntu.com*.archive.ubuntu.comppa.launchpat.netextras.ubuntu.com*.openvpn.netopenvpn.netgithub.com*.github.com496012525170.dkr.ecr.eu-central-1.amazonaws.comhub.docker.com*.hub.docker.compypi.python.org*.docker.comapi.snapcraft.io*.api.snapcraft.io*.githubusercontent.comgithubusercontent.compypi.org*.pypi.orgfiles.pythonhosted.org*.amazonaws.com*.ecr.eu-central-1.amazonaws.com*.api.ecr.eu-central-1.amazonaws.com*.eu-central-1.amazonaws.comamazonaws.com*.gcr.io*.registry-1.docker.io*.docker.iogcr.ioregistry-1.docker.iodocker.io*.googleapis.comstorage.googleapis.comp### Ubuntu default apt and snap repos*.ubuntu.comubuntu.com*.archive.ubuntu.comppa.launchpad.netextras.ubuntu.comapi.snapcraft.io*.api.snapcraft.io### Github to download the installation repositorygithub.com*.github.com*.githubusercontent.comgithubusercontent.com### AWS to download docker images and *.dkr.ecr.eu-central-1.amazonaws.com*.amazonaws.com*.ecr.eu-central-1.amazonaws.com*.api.ecr.eu-central-1.amazonaws.com*.eu-central-1.amazonaws.com*.s3.amazonaws.com### Docker to download additional container imageshub.docker.com*.hub.docker.com*.docker.com*.registry-1.docker.iodocker.io### Pypi repos to automate the installationpypi.python.orgpypi.org*.pypi.orgfiles.pythonhosted.org### Data & Moremtman.dataandmore.com (3.67.60.116)After installationĀ
The DMPP must have TCP access to mtman.dataandmore.com to enhance security monitoring capabilities. This access allows the DMPP to effectively monitor and analyze security-related data, ensuring that any potential threats or vulnerabilities are promptly identified and addressed. By maintaining a connection to mtman.dataandmore.com, the DMPP can proactively protect sensitive information and uphold compliance standards, ultimately contributing to a secure and resilient data environment.
Make sure that the DMPP has outgoingĀ TCP access to the following:https://mtman.dataandmore.com | 3.67.60.116https://*.s3.eu-central-1.amazonaws.com3.127.159.69 |Ā clientvpn.gdpr.dataandmore.com65.108.223.211 |Ā monitor.gdpr.dataandmore.com
Do not put a proxy server between the DMCS and the Internet unless you have a Custom Security Subscription.
Contact support@dataandmore.com for security information.
Certificate for Compliance Server
The compliance server needs a valid certificate for security reasons and seamless use of the notifications report. Please select a proper domain for the compliance server, such as; gdpr.yourorganisation.dk
Make sure that gdpr.yourorganisation.dk points to the compliance server's IP address.Create a .pfx type certificate using the registrant of your choice, and make sure to forward the password for the .pfx certificate to Data & More
Graph - Email - Postmark or SMTP Gateway
The DMCS needs access to a valid email account or an SMTP gateway to send reports to the end users. The email must be from the same domain as the users receiving it to reduce the risk of being flagged as spam.Ā For more information: https://support.dataandmore.com/en/knowledge/smtp
If you have links to open in the Outlook app, add the Reg edit
If the organisation uses Outlook as the mail client, it is possible to open emails directly from the GDPRtask board and see them in Outlook. Please run the Reg Edit script below on end-user computers to enable this. (http://woshub.com/how-to-create-modify-and-delete-registry-keys-using-gpo/)_________________________________________________
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\outlook]
@="URL:Outlook Folders"
"URL Protocol"=""
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\outlook\DefaultIcon]
@="C:\\Program Files\\Microsoft Office\\root\\Office16\\OUTLOOK.EXE"
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\outlook\shell]
@=""
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\outlook\shell\open]
@=""
[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\outlook\shell\open\command]
@="\"C:\\Program Files\\Microsoft Office\\root\\Office16\\OUTLOOK.EXE\" /select \"%1\""
Client Checklist for server installation.
Domain and pfx certificate + password has been sent to support@dataandmore.com
There is outgoing internet access on ports 443 and 80
DMZ is properly configured, and the proper ports are open/close
Ubuntu 24.04 /RedHat has been installed on the target server
The root partition on the D&MC Server has a minimum of 60 GB of free space
The root username and password for the D&MC Server have been given to D&M
A minimum of 500 GB disk has been mounted on D&MC Server
Any file shares that should be scanned are mounted on the D&MC Server under the path /mnt/data/file share/
The mounts have been added to the fstab
The domain gdpr.your company.local or gdpr.yourcompany.dk/com has been added to the company DNS. And the URL is pointing to the D&MC Server.
The Laptop has HTTPS access to the D&MC Server.