WARNING:tensorflow:Layer My_model Is Casting An Input Tensor From Dtype Float64 To The Layer's Dtype Of Float32, Which Is New Behavior In TensorFlow 2

Before my Tensorflow neural network starts training, the following warning prints out: WARNING:tensorflow:Layer my_model is casting an input tensor from dtype float64 to the layer

Solution 1:

tl;dr to avoid this, cast your input to float32

X = tf.cast(iris[:, :3], tf.float32) 
y = tf.cast(iris[:, 3], tf.float32)

or with numpy:

X = np.array(iris[:, :3], dtype=np.float32)
y = np.array(iris[:, 3], dtype=np.float32)

Explanation

By default, Tensorflow uses floatx, which defaults to float32, which is standard for deep learning. You can verify this:

import tensorflow as tf
tf.keras.backend.floatx()

Out[3]: 'float32'

The input you provided (the Iris dataset), is of dtype float64, so there is a mismatch between Tensorflow's default dtype for weights, and the input. Tensorflow doesn't like that, because casting (changing the dtype) is costly. Tensorflow will generally throw an error when manipulating tensors of different dtypes (e.g., comparing float32 logits and float64 labels).

The "new behavior" it's talking about:

Layer my_model_1 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2

Is that it will automatically cast the input dtype to float32. Tensorflow 1.X probably threw an exception in this situation, although I can't say I've ever used it.