How To Filter Na In R

Filtering Data with dplyr

Filtering data is one of the very basic performance when you work with data. You desire to remove a office of the data that is invalid or but you're not interested in. Or, yous want to zero in on a particular part of the data you want to know more about. Of course, dplyr has 'filter()' function to do such filtering, only at that place is fifty-fifty more than. With dplyr you tin can practice the kind of filtering, which could be hard to perform or complicated to construct with tools similar SQL and traditional BI tools, in such a uncomplicated and more than intuitive way.

Let's begin with some simple ones. Once again, I'll use the same flying data I have imported in the previous post.

Select columns

First, let's select columns that are interesting for at present. If you desire to know more than about 'how to select columns' please bank check this post I have written before.

          library(dplyr)          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME)

Filter with a value

Let's say you want to see only the flights of United Airline (UA). Y'all can run something like beneath.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
                          filter(CARRIER == "UA")

If yous want to use 'equal' operator you need to take two '=' (equal sign) together like in a higher place. If y'all run the above you'll see something like below.

And at present, let's observe the flights that are of United Airline (UA) and left San Francisco airport (SFO). You can use '&' operator as AND and '|' operator equally OR to connect multiple filter weather condition. This time we'll use '&'.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
                          filter(CARRIER == "UA" & ORIGIN == "SFO")

Or, y'all might want to see just the flights that left San Francisco airport (SFO) but are not of United Airline (UA). You tin use '!=' operator as 'non equal'.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
                          filter(CARRIER != "UA" & ORIGIN == "SFO")

Filtering with multiple values

What if you lot desire to run into only the data for the flights that are of either United Airline (UA) or American Airline (AA) ? You tin can use '%in%' for this, just similar the IN operator in SQL.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
            filter(CARRIER %in% c("UA", "AA"))

We can't really tell if it's working or not by looking at the commencement 10 rows. Allow'southward run count() function to summarize this rapidly.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
            filter(CARRIER %in% c("UA", "AA")) %>%
            count(CARRIER)

Nosotros can see merely AA and UA as nosotros expected. And yeah, I know, this 'count()' office is amazing. It literally does what you would intuitively imagine. It returns the number of the rows for each specified group, in this case that is CARRIER. We could have done this past using 'group_by()' and 'summarize()' functions, just for something similar this simple 'count()' office lonely does the job in such a quick manner.

Opposite the status logic

What if you want to see the flight that are non United Airline (UA) and American Airline (AA) this fourth dimension ? Information technology'south actually very unproblematic with R and dplyr. Here's a magic i alphabetic character you lot tin can use with any condition to reverse the effect. It's '!' (exclamation marker). And, it goes like this.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%
            filter(!CARRIER %in% c("UA", "AA")) %>%            
            count(CARRIER)

Notice that at that place is the exclamation mark at the kickoff of the condition inside the filter() function. This is a very handy 'function' that basically flips the effect of the condition that is afterward the exclamation mark. This is why the event above doesn't include 'UA' nor 'AA'. It might look a bit weird until you get used to it specially if you're coming from exterior of R world, but yous are going to run across this a lot and will capeesh its power and convenience.

Filtering out NA values

Now, let'southward go back to the original information once more.

          flying %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME)

When you look closer you'd notice that there are some NA values in ARR_DELAY cavalcade. You can get rid of them easily with 'is.na()' function, which would return TRUE if the value is NA and FALSE otherwise.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%            
            filter(is.na(ARR_DELAY))

Oops, information technology looks like all the values in ARR_DELAY are at present NA, which is opposite of what I hoped. Well, equally you saw already we can now try the '!' (assertion marker) function again like below.

          flight %>%            
            select(FL_DATE, CARRIER, ORIGIN, ORIGIN_CITY_NAME, ORIGIN_STATE_ABR, DEP_DELAY, DEP_TIME, ARR_DELAY, ARR_TIME) %>%            
            filter(!is.na(ARR_DELAY))

This is how you can work with NA values in terms of filtering the information.

This is the basic of how 'filter' works with dplyr. But this is just the beginning. You tin can do a lot more than by combining with aggregate, window, cord/text, and date functions, which I'chiliad going to cover at the next post. Stay tuned!