Problem 6. Basics for Spark Streaming and GraphX (Spark Streaming) Suppose we cr
ID: 3712236 • Letter: P
Question
Problem 6. Basics for Spark Streaming and GraphX (Spark Streaming) Suppose we create a Spark Streaming Context "streamingContext" in Python Spark. streamingContext- StreamingContext(sc, 3) where "c" is the SparkContext instance. What does the second parameter "3" represent here? Answer: We want to stream the data from TCP source with IP Address "123.45.67.89" and port number "4321". Please provide the function for streaming data from that TCP source. The return variable is a (Spark GraphX) Suppose we have a graph in the following. 4 The left figure shows the graph, and the right column shows the edge list file. Suppose the full directory of the file is "/home/rob/data/ToyGrapho1.txt". "SC is the Spark Context. Please provide the command to load the graph edge list file in Scala 1: If we want to get the number of vertices, we should use the following command in Scala 1:Explanation / Answer
Basically for answer 1 streamingcontext constructor defined here has two parameters and streamingcontext have various constructors for defining varios streamcontext here we have the following constructor:
StreamingContext(sc,3);
where sc is is the spark context instance and the generalized constructor is
StreamingContext(spark context Instance,batch duration)
hence 3 here defines -BATCH Duration
2) function for streaming data from TCP source
ssocketTextStream(localhost,port)
3)we use the following function load edges if sc is the spark context
sc.edges.map()
for vertices we can use
sc.vertices ()