{"id":20590,"date":"2026-07-13T04:42:51","date_gmt":"2026-07-13T04:42:51","guid":{"rendered":"https:\/\/www.infinitivehost.com\/blog\/?p=20590"},"modified":"2026-07-13T04:50:18","modified_gmt":"2026-07-13T04:50:18","slug":"setup-cuda-cudnn-dedicated-gpu-server-ubuntu-24","status":"publish","type":"post","link":"https:\/\/www.infinitivehost.com\/blog\/setup-cuda-cudnn-dedicated-gpu-server-ubuntu-24\/","title":{"rendered":"How to set up CUDA and cuDNN on..."},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"20590\" class=\"elementor elementor-20590\" data-elementor-post-type=\"post\">\n\t\t\t\t<div class=\"elementor-element elementor-element-fcdba03 e-flex e-con-boxed e-con e-parent\" data-id=\"fcdba03\" data-element_type=\"container\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-a4d8efd elementor-widget elementor-widget-heading\" data-id=\"a4d8efd\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\">How to set up CUDA and cuDNN on a dedicated GPU server (Ubuntu 24 guide)\n<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1152d64 elementor-widget elementor-widget-text-editor\" data-id=\"1152d64\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Installation of a server specifically designed to handle deep learning with the help of GPU is supposed to be easy but it seldom is. Problems with driver compatibility, incompatible versions, and incorrect CUDA path are some of the issues that everyone familiar with the process can relate to. The following tutorial will help you to install CUDA and cuDNN with minimum trouble on your Ubuntu 24 OS, irrespective of whether you have a local or remote server via InfinitiveHost or GPU4Host.<\/span><\/p><p><span style=\"font-weight: 400;\">Let&#8217;s get into it.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Why Your Server Location and Provider Matter<\/b><\/h2><p><span style=\"font-weight: 400;\">Before touching a single terminal command, your hosting choice shapes everything downstream. A dedicated GPU server in Germany behaves differently from one in the UK or Sweden \u2014 not just in latency, but in what software comes pre-configured.<\/span><\/p><p><span style=\"font-weight: 400;\">The <\/span><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-germany\"><span style=\"font-weight: 400;\">Germany GPU server Ubuntu 24 CUDA cuDNN ready<\/span><\/a><span style=\"font-weight: 400;\"> nodes from InfinitiveHost, for example, ship with NVIDIA drivers already validated against the kernel, which saves you a frustrating first hour of troubleshooting. Similarly, the <\/span><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-netherlands\"><span style=\"font-weight: 400;\">Netherlands GPU server CUDA cuDNN pre-configured node<\/span><\/a><span style=\"font-weight: 400;\"> eliminates the manual driver installation step entirely.<\/span><\/p><p><span style=\"font-weight: 400;\">If you&#8217;re working on AI research or production model training, here&#8217;s a quick rundown of where providers are placing GPU infrastructure right now:<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-uk\"><b>UK GPU dedicated server<\/b><\/a><span style=\"font-weight: 400;\">: cuDNN Ubuntu 24 setups with strong European latency<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-france\"><b>France GPU node<\/b><\/a><span style=\"font-weight: 400;\">: CUDA toolkit Ubuntu installations with OVH-peered connectivity<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-sweden\"><b>Sweden GPU server<\/b><\/a><span style=\"font-weight: 400;\">: cuDNN deep learning environments with green energy infrastructure<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-switzerland\"><b>Switzerland GPU server<\/b><\/a><span style=\"font-weight: 400;\">: secure CUDA dev environments, strong for privacy-sensitive AI work<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-ireland\"><b>Ireland GPU server<\/b><span style=\"font-weight: 400;\">:<\/span><\/a><span style=\"font-weight: 400;\"> Ubuntu CUDA AI workload setups with excellent AWS Transit peering<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-cloud-server-india\"><b>India GPU cloud<\/b><span style=\"font-weight: 400;\">:<\/span><\/a><span style=\"font-weight: 400;\"> Ubuntu 24 CUDA development servers for the Asia-Pacific market<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><a href=\"https:\/\/www.infinitivehost.com\/gpu-dedicated-server-usa\"><b>USA GPU server<\/b><\/a><span style=\"font-weight: 400;\">: CUDA 12 cuDNN Ubuntu 24 setups for North American deployments<\/span><\/li><\/ul><p><span style=\"font-weight: 400;\">Worth mentioning:<\/span><a href=\"http:\/\/www.infinitivehost.com\"><span style=\"font-weight: 400;\"> InfinitiveHost CUDA-ready GPU servers<\/span><\/a><span style=\"font-weight: 400;\"> are currently available at 25% OFF, which is a solid entry point if you&#8217;re provisioning for a long-term project.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Step 1: Verify Your GPU and System Environment<\/b><\/h2><p><span style=\"font-weight: 400;\">SSH into your dedicated GPU server and start with a clean system audit.<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">lspci | grep -i nvidia<\/span><\/p><p><span style=\"font-weight: 400;\">uname -r<\/span><\/p><p><span style=\"font-weight: 400;\">cat \/etc\/os-release<\/span><\/p><p><span style=\"font-weight: 400;\">You&#8217;re looking for Ubuntu 24.04 LTS confirmed, and your NVIDIA GPU visible in the PCI list. If the GPU doesn&#8217;t show, it&#8217;s a driver or hardware recognition issue \u2014 resolve that before proceeding.<\/span><\/p><p><span style=\"font-weight: 400;\">Check whether any NVIDIA drivers are already installed:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">nvidia-smi<\/span><\/p><p><span style=\"font-weight: 400;\">If this returns a valid output with driver version and GPU details, you&#8217;re partway there. If it throws an error, you&#8217;ll install drivers from scratch in the next step.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Step 2: Install NVIDIA Drivers<\/b><\/h2><p><span style=\"font-weight: 400;\">Update your package index first:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt update &amp;&amp; sudo apt upgrade -y<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt install -y ubuntu-drivers-common<\/span><\/p><p><span style=\"font-weight: 400;\">sudo ubuntu-drivers autoinstall<\/span><\/p><p><span style=\"font-weight: 400;\">sudo reboot<\/span><\/p><p><span style=\"font-weight: 400;\">After reboot, confirm the driver is loaded:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">nvidia-smi<\/span><\/p><p><span style=\"font-weight: 400;\">You should see your GPU model, driver version, and CUDA version listed. On a properly configured dedicated GPU server, this step is often already done for you \u2014 but it&#8217;s worth verifying.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Step 3: Install the CUDA Toolkit<\/b><\/h2><p><span style=\"font-weight: 400;\">For Ubuntu 24, CUDA 12.x is the current production-stable release. Head to the NVIDIA CUDA repository and add it manually:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">wget https:\/\/developer.download.nvidia.com\/compute\/cuda\/repos\/ubuntu2404\/x86_64\/cuda-keyring_1.1-1_all.deb<\/span><\/p><p><span style=\"font-weight: 400;\">sudo dpkg -i cuda-keyring_1.1-1_all.deb<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt update<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt install -y cuda-toolkit-12-4<\/span><\/p><p><span style=\"font-weight: 400;\">Once installed, add CUDA to your PATH. Open your <\/span><span style=\"font-weight: 400;\">.bashrc<\/span><span style=\"font-weight: 400;\">:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">nano ~\/.bashrc<\/span><\/p><p><span style=\"font-weight: 400;\">Add all these below-mentioned lines at the bottom:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">export PATH=\/usr\/local\/cuda\/bin:$PATH<\/span><\/p><p><span style=\"font-weight: 400;\">export LD_LIBRARY_PATH=\/usr\/local\/cuda\/lib64:$LD_LIBRARY_PATH<\/span><\/p><p><span style=\"font-weight: 400;\">Source it:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">source ~\/.bashrc<\/span><\/p><p><span style=\"font-weight: 400;\">Check out installation:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">nvcc &#8211;version<\/span><\/p><p><span style=\"font-weight: 400;\">You should see CUDA 12.x confirmed. This is the core of what makes a dedicated GPU server useful for model training \u2014 without a working CUDA toolkit, the GPU is just sitting idle.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Step 4: Install cuDNN<\/b><\/h2><p><span style=\"font-weight: 400;\">cuDNN is what neural network frameworks like TensorFlow and PyTorch actually use for optimized GPU computation. NVIDIA has simplified the installation process considerably for Ubuntu 24.<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt install -y libcudnn9-cuda-12<\/span><\/p><p><span style=\"font-weight: 400;\">Or for cuDNN 8 (required by some older TensorFlow versions):<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt install -y libcudnn8 libcudnn8-dev<\/span><\/p><p><span style=\"font-weight: 400;\">Verify cuDNN is installed correctly:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">dpkg -l | grep cudnn<\/span><\/p><p><a href=\"https:\/\/www.gpu4host.com\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">GPU4Host CUDA <\/span><\/a><span style=\"font-weight: 400;\">environment server configuration tips recommend pinning your cuDNN version to match your framework. TensorFlow 2.13+ works cleanly with cuDNN 8.9 on CUDA 12, while PyTorch 2.x supports cuDNN 9.x natively.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Step 5: Validate the Full Stack<\/b><\/h2><p><span style=\"font-weight: 400;\">Run a fastest Python validation to validate CUDA and cuDNN are talking to each other in the right way:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">python3 -c &#8220;import torch; print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0))&#8221;<\/span><\/p><p><span style=\"font-weight: 400;\">For TensorFlow:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">python3 -c &#8220;import tensorflow as tf; print(tf.config.list_physical_devices(&#8216;GPU&#8217;))&#8221;<\/span><\/p><p><span style=\"font-weight: 400;\">Both should return your GPU name and validate CUDA access. If you&#8217;re on an InfinitiveHost CUDA-ready node or a USA GPU server CUDA 12 cuDNN Ubuntu 24 setup, this validation step typically passes cleanly without additional troubleshooting.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Common Issues Worth Knowing<\/b><\/h2><p><span style=\"font-weight: 400;\">Driver and CUDA version mismatch is the single most common failure point. Each CUDA version requires a minimum driver version. CUDA 12.4 needs driver 550.54 or higher.<\/span><\/p><p><span style=\"font-weight: 400;\">Kernel header mismatch happens after system updates. If <\/span><span style=\"font-weight: 400;\">nvidia-smi<\/span><span style=\"font-weight: 400;\"> breaks after an apt upgrade, reinstall the DKMS module:<\/span><\/p><p><span style=\"font-weight: 400;\">bash<\/span><\/p><p><span style=\"font-weight: 400;\">sudo apt install -y dkms nvidia-dkms-550<\/span><\/p><p><b>Wrong Python environment<\/b><span style=\"font-weight: 400;\"> \u2014 always activate your virtual environment before testing. System Python and conda environments don&#8217;t share CUDA paths.<\/span><\/p><p><span style=\"font-weight: 400;\">On any well-configured <\/span><b>dedicated GPU server<\/b><span style=\"font-weight: 400;\"> \u2014 whether it&#8217;s a Switzerland GPU server secure CUDA dev environment or an Ireland GPU server Ubuntu CUDA AI workload setup \u2014 these issues are minimized because the base kernel and driver pairing is pre-validated.<\/span><\/p><h2 style=\"font-size: 24px; margin-top: 20px;\"><b>Conclusion<\/b><\/h2><p><span style=\"font-weight: 400;\">Getting CUDA and cuDNN running cleanly on a dedicated GPU server comes down to three things: the right provider, the right version pairing, and the right installation order. Ubuntu 24 is stable and well-supported by NVIDIA&#8217;s current toolkit, making it the right base OS for serious AI work in 2026.<\/span><\/p><p><span style=\"font-weight: 400;\">Whether you&#8217;re running a France GPU node CUDA toolkit Ubuntu installation, a Sweden GPU server cuDNN deep learning environment, or starting fresh on a USA GPU server CUDA 12 cuDNN Ubuntu 24 setup \u2014 this guide gets you from zero to a verified working environment. For managed setups with pre-configured drivers, InfinitiveHost CUDA-ready GPU servers at 25% OFF are worth serious consideration for teams who&#8217;d rather spend time training models than debugging kernel modules.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p><span class=\"elementor-category-label\"><a href=\"https:\/\/www.infinitivehost.com\/blog\/category\/gpu-dedicated-server\/\">GPU Dedicated Server<\/a><\/span>How to set up CUDA and cuDNN on a dedicated GPU server (Ubuntu 24 guide) Installation of a server specifically designed to handle deep learning with the help of GPU is supposed to be easy but it seldom is. Problems with driver compatibility, incompatible versions, and incorrect CUDA path are some of the issues that [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":20598,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[331],"tags":[],"class_list":["post-20590","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-gpu-dedicated-server"],"_links":{"self":[{"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/posts\/20590","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/comments?post=20590"}],"version-history":[{"count":7,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/posts\/20590\/revisions"}],"predecessor-version":[{"id":20597,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/posts\/20590\/revisions\/20597"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/media\/20598"}],"wp:attachment":[{"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/media?parent=20590"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/categories?post=20590"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.infinitivehost.com\/blog\/wp-json\/wp\/v2\/tags?post=20590"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}