[coursera/SequenceModels/week2]NLP&Word Embeddings (summary&question)

Suppose you learn a word embedding for a vocabulary of 10000 words. Then the embedding vectors
should be 10000 dimensional, so as to capture the full range of variation and meaning in those
words.

False

Correct

A non-linear dimensionality reduction technique

Correct

Suppose you download a pre-trained word embedding which has been trained on a huge corpus of text.
You then use this word embedding to train an RNN for a language task of recognizing if someone is
happy from a short snippet of text, using a small training set.

x (input text)	y (happy?)
I'm feeling wonderful today!	1
I'm bummed my cat is ill.	0
Really enjoying this!	1

Then even if the word “ecstatic” does not appear in your small training set, your RNN might
reasonably be expected to recognize “I’m ecstatic” as deserving a label y=1>.

True

False

e
boy−
e
girl
≈
e
brother
−
e
sister

e
boy
−
e
girl
≈
e
sister
−
e brother

e boy − e brother ≈ e girl
− e sister

eboy−ebrother≈esister−egirl

It is computationally wasteful.

The correct formula is
ET∗
e
1234

.

This doesn’t handle unknown words (<UNK>).

None of the above: Calling the Python snippet as described above is fine.

True

False

In the word2vec algorithm, you estimate P(t∣c), where
t is the target word
and c is a context
word. How are t and
c chosen from the
training set? Pick the best answer.

c is a
sequence of several words immediately before
t.

c and
t are
chosen to be nearby words.

c is the
sequence of all the words in the sentence before
t.

Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings. The
word2vec model uses the following softmax function:

P(t∣c)=eθTtec∑10000t′=1eθTt′ec

Which of these statements are correct? Check all that apply.

θt
and
ec
are both 500 dimensional vectors.

θt
and
ec
are both 10000 dimensional vectors.

θt
and
ec
are both trained with an optimization algorithm such as Adam or gradient descent.

After training, we should expect
θt
to be very close to
ec
when t
and c are the same word.

Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings.The GloVe
model minimizes this objective:

min
∑
10,000i=1∑
10,000j=1
f(
X
ij

)(
θTiej+
bi+
b′j−log
X
ij)

2

Which of these statements are correct? Check all that apply.

θi and ej should be initialized to 0 at the beginning of
training.

θi and ej should be initialized randomly at the beginning of training.

Xij is the number of times word i appears in the context of word j.

The weighting function f(.) must satisfy
f(0)=0 .

m1 >>
m2

m1 <<
m2