Conda + cuda + cudnn + tensorflow-GPU + keras-GPU

1) nvidia-smi


2) conda create --name tf1-gpu

3) conda install tensorflow-gpu==1.15.0 ## will auto-install cuda and cudnn for version of nvidia-smi

4) conda install keras-gpu



Question:

1)
==>Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

from keras.backend.tensorflow_backend import set_session
import tensorflow as tf
config = tf.ConfigProto()
### dynamically grow the memory used on the GPU
config.gpu_options.allow_growth = True
### to log device placement (on which device the operation ran)
config.log_device_placement = True
### set this TensorFlow session as the default session for Keras
sess = tf.Session(config=config)
set_session(sess) 

控制GPU资源使用率

# 使用allow_growth option，刚一开始分配少量的GPU容量，然后按需慢慢的增加，由于不会释放

#内存，所以会导致碎片


2) Tensorflow中tf.ConfigProto()详解

import tensorflow as tf

session_config = tf.ConfigProto(

      log_device_placement=True,

      inter_op_parallelism_threads=0,

      intra_op_parallelism_threads=0,

      allow_soft_placement=True)

sess = tf.Session(config=session_config)

————————————————

当使用GPU时候，Tensorflow运行自动慢慢达到最大GPU的内存

session_config.gpu_options.allow_growth = True

当使用GPU时，设置GPU内存使用最大比例

session_config.gpu_options.per_process_gpu_memory_fraction = 0.4

是否能够使用GPU进行运算

tf.test.is_built_with_cuda()

————————————————

For Tensorflow2:

use tf.keras

Question:
Error : Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above

def solve_cudnn_error():
    gpus = tf.config.experimental.list_physical_devices('GPU')
    if gpus:
        try:
            # Currently, memory growth needs to be the same across GPUs
            for gpu in gpus:
                tf.config.experimental.set_memory_growth(gpu, True)
            logical_gpus = tf.config.experimental.list_logical_devices('GPU')
            print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
        except RuntimeError as e:
            # Memory growth must be set before GPUs have been initialized
            print(e)
           
solve_cudnn_error()