Case 1: Application to Recommendation System

First let’s import the necessary libraries

library(NNLM)
library(rNMF)

Just as a brief reminder NMF takes the original data (\(X\), say \(n \times p\)) and decomposes it to a matrix \(W\) and a matrix \(H\), so as \(X = W H\). Dimensions of \(W\) is (\(n \times r\)) and dimensions of \(H (r \times p)\). r is the “hidden dimensions” that we aim to discover and represent the underlying structure of our data.

Suppose that we have a very simple case of 5 users ratings on 4 movies. The following code summarizes the data. Notice that there are also some missing values.

movies=c("Top Gun", "Rambo", "Young Guns", "Lonesome Dove")
x1 <- c(5,4,1,1)
x2 <- c(4,5,NA,1)
x3 <- c(1,1,5,5)
x4 <- c(1,1,4,5)
x5 <- c(NA,1,5,4)

R <- as.matrix(rbind(x1,x2,x3,x4,x5)) # n = 5 rows p = 4 columns 
colnames(R)=movies

Our goal in matrix facrorization is two-fold. First, to discover the hidden dimensions (r) and second be able to predict missing ratings (e.g. the NA values on the matrix).

Let’s apply the NMF method on the R matrix and select r=2 (hidden dimensions).

set.seed(12345)
res <- nnmf(R, 2, method="lee") # lee & seung method
res

Inspect the results of the factorization. There are several evaluation measures (like MSE) and performance information (running time). Now let’s see the parts of the result and more specifically matrices W and H.

w <- res$W #  W  user feature matrix matrix
dim(w) # n x r (n= 5  r = 3)
print(w) 

h <- res$H # H  movie feature matrix
dim(h) #  r x p (r = 3 p = 4)
print(h)

By looking at matrix W, do you see any pattern regarding the two hidden dimensions? Could we seperate users based on this information? Similarly, can we do the same for matrix H (i.e. could we seperate movies based on this information)?

Write some code to predict the ratings based on these matrices. More specifically, let’s predict the rating of user 2 (x2) for movie Guns. Consult the lecture slides for the exact formulation (although should be pretty easy). Hint: From w matrix you need user #2 (i.e. 2-th row ) and from h matrix you need movie #3 (i.e. the 3-rd column). You can multiply vectors (or arrays) using the \(\%*\%\) operator.

Then, let’s try to reproduce the whole input

x <- w %*% h
print(x)

How does it look? Were we also able to predict the values that were not existent? Do they make sense?

As an extra step, try to replace another value of user 4 with NA, so make the entry (5,2) of your array also NA (i.e. have predictions only for the two last movies for user 5). What’s the result now?

Representation of vectors

Now let’s try to represent the hidden dimensions in a plot and see what was discovered by NMF.

movies1 <- data.frame(t(h))
features1 <- cbind(movies1$X1,movies1$X2)
plot(features1)
title("Movie Feature Plot")

What does this plot show? How are the four movies spread?

Case 2: NMF for image reconstruction

First we load the build-in data set “Symbols_c” (in rNMF package), a 5625 by 30 matrix where each column contains a vectorized 75 by 75 image.

data(Symbols_c)
see(Symbols_c, title = "Corrupted data set")

Let’s apply NMF with k=4 (and some additional parameters as well that we won’t deal with now)

res <- rnmf(Symbols_c, k = 4, gamma = 0.03, showprogress = FALSE, my.seed = 100, tol = 0.0001, maxit = 50)

Let’s see what are the basis vectors (matrix W) that NMF decomposes data to.

see(res$W, title = "Regular NMF basis", layout = c(1,4))

Shadows in above images indicate that corruptions were not removed and contaminated the decomposition. However, NMF successfuly finds that basis of all images is the 4 basic shapes.

Now let’s also see how it reconstructs the original images.

see(res$fit, title = "Regular NMF reconstruction with k = 4")

Can you compare it with the initial one? Is it better?

Also, we have the option of detecting the outliers, as below.

outliers <- matrix(0, nrow = nrow(Symbols_c), ncol = ncol(Symbols_c))
outliers[res$trimmed[[res$niter]]] <- 1
see(outliers, title = "Outliers extracted by rNMF")

Case 3: Reconstruction of image for noise removal

In this example we see the compression of a single corrupted image by both regular NMF and rNMF. First we load a build-in corrupted face image and let’s look at it.

data(face)
see(face, title = "Corrupted face image", col = "grey", input = "single")

I hope you can clearly see the noise :)

Let’s apply NMF and see the result.

res2 <- rnmf(face, k = 50, gamma = 0.025, showprogress = FALSE, my.seed = 100)
see(res2$fit, title = "rNMF compression", col = "grey", input = "single")

Much better isn’t it? However, the quality is dropped. Why? Experiment with the value of k.