I am trying to insert into an existing mysql table using Pyspark JDBC connection however I get the following error(picture attached)
Can I get assistance on this error. The table exist in the MySql Database, I was successful in Inserting with a Dataframe when selecting data form another table , however i would like to Insert directly to the Mysql datadase table.
Below is the Pyspark code I am using to insert into the table:
import os
from pyspark.sql import SQLContext
from pyspark import SparkContext
from pyspark.sql import SparkSession
import logging
sc = SparkSession.builder.appName("TestConnection").enableHiveSupport().getOrCreate()
sqlContext = SQLContext(sc)
Provide your Spark-master node below
hostname = "localhost"
dbname = "dev_db"
username = "****"
password = "*****"
jdbc_url = "jdbc:mysql://{0}/{1}?user={2}&password={3}".format(hostname,dbname,username,password)
sqlContext.sql("insert into test values ('3','Names',123654)").write.format('jdbc').options(url=jdbc_url,driver='com.mysql.jdbc.Driver').mode(append).save()
logging.info("*********Write to MySql Completed*********")
