RUNNING HEAD: Student Staff Selection! !

"!

Loyola University Chicagos Department of Residence Life Student Staff Selection: Evaluation Plan Tyeishia Banks and Crystal Norwood

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Table of Contents Introduction ........................................................................................................... 3 Description of Loyola University Chicago ..................................................... 3 Description of the Resident Life Program at Loyola ..................................... 4 Description of the Student Staff Selection at Loyola...................................... 5 Program Description ....................................................................................... 6 Student Staff Selection Goals and Mission..................................................... 6 Stakeholder Analysis ...................................................................................... 9 Literature Review............................................................................................10 Logic Model Description ......................................................................................11 External Factors ..............................................................................................14 Evaluation Design .................................................................................................17 Purpose............................................................................................................15 Goal .................................................................................................................15 Quantitative Approach .........................................................................................17 Survey Population ..........................................................................................17 Survey Design .................................................................................................17 Implementation ...............................................................................................17 Pilot Testing ...................................................................................................19 Statistical Analysis ..........................................................................................20 Qualitative Approach ...........................................................................................21 Focus Group Plan ...........................................................................................22 Focus Group Implementation ........................................................................25 Focus Group Analysis ....................................................................................26 Focus Group Limitations ...............................................................................27 Final Report ...................................................................................................28 References ..............................................................................................................30 Appendices .............................................................................................................A-L Appendix A: Logic Model ......................................................................................31 Appendix B: Survey Matrix ...................................................................................33 Appendix C: Survey................................................................................................38 Appendix D: Focus Group Protocol .......................................................................43 Appendix E: Focus Group Note Sheet ....................................................................46 Appendix F: Informed Consent ..............................................................................49 Appendix G: A Prior Construct Map .....................................................................51 Appendix H: Student Staff Selection Timeline ......................................................52 Appendix I: Budget ...............................................................................................53 Appendix J: Survey Communication ......................................................................54 Appendix K: Focus Group Communication ...........................................................56 Appendix L: Final Presentation ..............................................................................59

1!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Introduction

2!

As part of the Residence Life community that strives to serve diverse identities, several student populations are absent and overlooked. This student population includes students of color. Diversifying the student applicant pool is crucial to the department because they recognize the essential role that diversity plays in educational excellence and developing communities. The department of Residence Life is dedicated to recruiting, hiring, selecting, promoting, and retaining a competent and diverse staff that reflects the cultural and personal diversity of Loyola and greater Chicago community. We are concerned about the issue and are exploring it in the evaluation process due to the lack of applicants of color applying for positions. The problems surrounding the Selection Process focus on unsuccessful recruitment strategies. The recruitment process and strategies have shown to recruit great student staff leaders, but show a small amount of diversity in the selection pool. We recognize the issues with recruiting and retaining specific student populations, but have not found sufficient ways to use this information for the student staff selection process. This process recruits students for the resident assistants (RA) and learning community assistants (LCA). Discussions within the department are on stigmas, areas of discomfort, and assumptions when making the final placement decision. Another stigma focuses on the impact of a small professional staff identifying as people of color, and the effect they have on students of color during the selection process. An area of discomfort that is most apparent during the placement process when placing students of color in the LUC residential communities. As a department they gather valuable information pertaining to the student staff population but are not successful in implementing this results for the improvement of the selection process. The end results are the

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

3!

completion of the selection process, along with placement and tokenizing the limited amount of student of color as the diversity in each residence hall. Loyola University Chicago Loyola University of Chicago is a four-year, coeducational, private research university located in Chicago, Illinois. Loyola is a major piece of the city of Chicago and it is comprised of four campuses in the Chicago area, and three international campus centers (Rome, Beijing, and Vietnam). It was established in 1870 and is affiliated with the Roman Catholic Church, and is the largest Jesuit University in the country (Loyola University Chicago, 2013). The institutions mission states We are Chicago's Jesuit Catholic University- a diverse community seeking God in all things and working to expand knowledge in the service of humanity through learning, justice and faith(Loyola University Chicago, 2013). Loyolas mission is to the whole person and supporting the lifelong professional development that is demonstrated through all departments on campus. Loyola University Chicago students, staff, and faculty appear to benefit from Loyolas Jesuit education and values. Loyola University Chicago Jesuit values allude to seeking knowledge in the service of humanity; preparing to live in a shared, global community; and building on the desire to always do and become more. Loyola University Chicagos Jesuit tradition distinguishes Loyola University Chicago from other colleges and universities. Loyola values the view of pursuit of knowledge, the embracing of faiths, and the promotion of justice as intrinsically related. Department of Residence Life Loyola's Department of Residence Life manages all residence halls for the on-campus student population of 4500 plus students. The department oversees and operates a total of twenty

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

4!

residence halls including apartments; nineteen located in the Rogers Park and Edgewater neighborhoods surrounding the Lake Shore Campus and one at the Water Tower Campus. The vision of the department is to foster development and to support the vision, which states Our residential communities prepare people to lead extraordinary lives by integrating learning, spiritual growth and development. We contribute to students transformative education by offering student centered programs, services and environments that foster student involvement, responsibility and leadership (Loyola University Chicago, 2013). Loyola University Chicago, Department of Residence life serves a multi- dimensional population, which includes: learning communities, first year communities, upper-class communities, transfer students, and graduate students. At Loyola University Chicago the mission aligns and coincide with the departments vision. One of the departments core values is diversity. It states, We value civility, respect, appreciation, and an in-depth understanding of diversity including the ability to put knowledge into meaningful action (Loyola University Chicago, 2013).! The mission of the Residence Life Department focuses on enhancing the campus experience by creating transformative environments by providing safe, secure residence halls and inclusive communities where students, staff, and faculty integrate key academic, social, spiritual, and recreational experiences. The overall commitment is to ignite the passions and social responsibility of every individual in partnership with our residents (Loyola University Chicago, 2013). The goal of the Department of Residence Life is to form a cohesive 100+ Student Staff for the following academic year. The department works to create a diverse and mission driven identity each year. Positions offered by the department include but are not limited to Resident Assistants, Furniture Crew, Student Office Assistants, Desk Managers and Receptionist. These

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

5!

positions are highly valued because each serves as unique contributor to the student experience. Each position serves as a peer to peer resource and assist in the development of the community at Loyola University Chicago. Student Staff Selection The Student Staff Selection process is a leadership opportunity offered to students who are looking for opportunities of meaningfully contribution to the residential community at Loyola University Chicago. Loyola University Chicago (LUC) Department of Residence Life initiates this opportunity. The Student Staff Selection process intends to recruit an estimated 100 students across different disciplines, identities, and class standings. Student Staff Selection engages and allows students to deepen their own development personally or professionally whether or not chosen for the position. As mentioned previously the mission of the Department of Residence Life supports this process of student engagement in during student staff selection. Part of the department mission is that they provide integrated learning, which supports the group process students go through during selection. Another part of the mission is that the selection process supports student development. In the process the department provides opportunities of professional development through one on one interview process. The importance of the student leader positions is reinforced through Group Process. Group Process is a two-day, in-depth process that captures group interactions. During the group process the Department of Residence Life observes each candidate's knowledge and understanding of a variety of competences, communication and leadership skills, individual and group manners, and participation as they interact as a group to address each prompt. Through this process students understand that through training and accessible resources that can have an impact on students.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

6!

Through these positions Resident Assistants become agents for change and allies for their communities and other student populations. Through this process the Department of Residence Life also hope students understand the departments mission and values. As a department they aim to hire strong leaders they can ultimately be a resource for different identities such as sexual orientation, socioeconomic status, first generation, religion but not limited to racial and ethnic identities. The Student Staff Selection process has played a dynamic role in developing the student leadership community, and has created opportunities for leadership among the growing student populations. With the selections process serving as a recruitment tool for the Department of Residence Lifes evolving and changing culture, the selection process is forever changing. As the department makes changes to better assist the needs of the current student population, the selection process will need to continue make the similar changes. Currently the Department of Residence Life lacks the archive of the history of their selection process prior to 2010. Currently we have a new Director but not new to the institution or department. Prior to him accepting the Director position, the former director and professionals within the department started to implement structure with the selection process. Since then there has been no archiving of materials to evaluate and improve the program. The student Staff Selection has operated without official written goals, objectives, or learning outcomes stated for the overall process (Katie Rutkowski, personal communication, October 2013). The department has operated under the impression that staff understands the goals and purpose of Student Staff Selection. The Department of Residence Life every Spring Semester implements a selection process and has operated each year without documenting goals, objectives and learning outcomes. In speaking with the Assistant Director, the Student Staff Selection has changed over

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

7!

the years, whether adding positions, revamping group process, or even the marketing strategies (Katie Rutkowski, personal communication, October 2013). As a result, challenges and success have developed. Student Staff Selection typically starts in February and last until late March ending with the filling of positions. In order to prepare for student staff selection, the department specifically created a Selection Committee who is comprised of Graduate Assistants within the department and Professional Staff members (Resident Directors, Assistant Directors and Area Coordinators). The committee is tasked with creating the timeline for student staff selection, overall group process, interview breakouts, marketing, hiring, and assessing the overall process. During the recruitment process the Department of Residence Life focuses on cost efficient marketing, and forms of outreach that encourages other partners to get involved with the selection process. Previous years advertisements that were used during the recruitment included forms of tabling with current student staff, items such as coffee sleeves based on the culture of the campus, informational sessions, and informational flyer. The committee is created by mid-September and meets and average of three times a month to prepare for the spring. Being that the student staff selection process committees primary focus it a time to prepare for the high traffic period of the year. As members of the committee in 2012-2013, we (Crystal and Tyeishia) have been able to develop some general themes that we believe can become learning outcomes for Student Staff Selection. The following outcomes are listed below: ! Students able to better articulate the departments values and mission. ! Students will be able to experience a professional interview process to deepen their skills and being able to articulate their experience.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

8!

! Students will be able to deepen their understanding of the position applied for and become knowledgeable further in regards to the process. ! Returning student staff will be able to better articulate how they have grown throughout there time in their position as well as how they have changed. Importance: Stakeholders Stakeholders are important in order to maintain a strong collaboration and united front as a part of the Division of Student Development. They are the agents that will implement the changes that any office goes through and will be able to provide their insight on best practices. Wholey, Hatry, and Newcomer (2010) define stakeholders as individuals, groups, or organizations that can effect or are affected by an evaluation process or its findings (p.31). Direct stakeholders are those individuals that have a direct or immediate stake in the program while indirect stakeholders are those individuals who might indirectly benefit from the success or improvement of the program. Direct & Indirect Stakeholders For the means of this evaluation the direct stakeholders include the Residence Life staff, graduate assistants and participants involved in the process. The firsts sets of stakeholders are the students, which estimate about two hundred. These are students who have been selected to move on in the selection process, upon submitting their application. The students who are hired are the front line of communication and strive to create a safe and secure atmosphere for the residents within the residential communities. The students that are hired serve as an academic support, policy enforcers, community builders, program implementers and role models for their community. Another pair of stakeholders are the individuals that oversee activities and

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"9!

interviews during the selection process. These individuals are Residence Life staff that will eventually oversee and work of the student leaders throughout the year. Each of these individuals can be seen as direct stakeholders because of their involvement. Without them the selection process could not be implemented. Primary intended users are stakeholders within our evaluation project. Primary intended users are specific stakeholders selected to work with the evaluator throughout the evaluation to focus the evaluation, participate in making design and method decisions, and interpret the results to assure that the evaluation is useful, meaningful, relevant, and credible (Wholey et al., 2010, p.31). The primary user identified in this evaluation approach is Katie Rutkowski who is a full time professional in the department committed to this selection process. Other groups affected by the evaluation approach are indirect stakeholders. Within this evaluation approach the indirect stakeholders are the residents that make up the on-campus residential population. This comes into effect because the department needs to hire a diverse student staff to meet the needs of our students, department and institution. The residents and the surrounding Loyola community benefit because its an opportunity for students to seek out a mentor. The investment that stakeholders have in an office or particular programs also allows for them to influence other stakeholders and share their thoughts and experiences to provide new suggestions (Wholey, Hatry, & Newcomer, 2010). As previously mentioned there are several stakeholders throughout the Student Staff Selection Process, and to ensure each student staff learns the information intended for them the stakeholders must be present. Literature Review

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

""!

The conceptual framework for the Student Staff Selection process has originated in several articles and documents. Alimo et al (2002) stated that students believe that there most significant interactions with those racially and/ or ethnically different from themselves were in residence hall. This supports our evaluation concern of seeking a diverse student staff population to foster and support these future conversations. Other articles used were Student Perceptions of Campus Cultural Climate by Race. This article suggest that by understanding of the particular experiences and perceptions of African American, Asian American, Latino/a, and White college students may also influence the development of culturally relevant and effective interventions (Ancis, Sedlacek, Mohr, 2002). The Cultural Attitudes and Climate Questionnaire (CACQ), discussed in Ancis, Sedlacek, and Mohr (2002) measures students perceptions and experiences of the university racial and ethnic climate and advocates for the Department of Residence Life mission and values. As the Student Staff Selection process continues the department will need to continue to benchmark against other insinuations and develop its philosophy to dive their belief of diversity and community in the department. Logic Model Description The structure of the Student Staff Selection process at LUC is described in the logic model, found in Appendix A. A logic model starts with long-term visions of how participants will be better off because of a program (Fitzpatrick, 2004, p.70). The model transitions from resources, to inputs, to outcomes as illustrated in the logic model. THIS IS WHERE I STOPPED Inputs are described as the resources the Department of Residence Life invests in the Student Staff Selection. Fitzpatrick (2004) identifies inputs as annual budgets, staffing facilities, and materials, needed to run a program. The most critical

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"1!

individuals to this process are the departments selection committee who has coordinated all aspects of the Student Staff Selection process. In addition there are outside staffs that are not members on the department committee, which include Assistant Resident Directors (ARD), Resident Directors (RD), Area Coordinators (AC), Associate Directors (AD) and the Director who plays an indirect role. These members take in consideration other inputs such as recruitment strategies, funds allocated for resources (printing, room reservation, food), and student staff necessities. Outputs describes what takes place within the Student Staff Selection process and who participates in the selection process. Fitzpatrick (2004) suggest output as including the number of participants each week by demographic category, the number of class meetings, and even hours of direct service to each participant (p.79). Some outputs are team builders, icebreakers, selection social, group process, and interviews. Outcomes describe the impact of the program in the LUC community and department, with both short and long term goals. To achieve these goals certain assumptions must be made of the Student Staff Selection process and the participants. Additionally there are factors that should be considered when implementing this process each year. However the following sections below provide a more detailed understanding of these sections of the logic model (Appendix A). Outcomes The outcome structure shows the benefits of the participants and the organization involved. These short-term goals are intended to lead to medium goals. Some short-term goals are giving students the ability to learn and explore the positions with deeper understanding pass the basic compensation provided. Secondly, to be able to understand the selection process and developmental structures put in place such as the interview. The interview structure use by the selection committees mimics a professional interview process. Moving from short-term goals to

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"2!

medium goals within the logic model (Appendix A) one goal is to assess where students are at in supporting the departments mission and values. Additionally, another goal is create comfort within other students and staff. The medium goals are ultimately within the process to lead to a long-term goal. The ultimate long-term goal is to hire a cohesive student staff that can articulate their values and what it means to live out the departments values and ideals. Another long-term goal is to hire a student staff that is reflective of the student population at LUC. An important longterm goal is that students will be equipped with knowledge to approach any situation and the skills and techniques to build on their own development and articulate these capabilities on a resume. Assumptions As we approach this evaluation project we see importance in recognizing that certain assumptions will be made. These assumptions whether they are intentional or unintentional may influence the outcomes. We acknowledge that we bring our biased perspective when approaching the project as previous members of the selection committee and current staff of the department. Our logic model includes assumptions that influence our evaluation. First, we assume that students will be able to benefit from the group process session. Although every student anticipates to be hired, we assume if a student does not receive a position that they will reapply next year or apply for a different staff position within the department. We also assume that students will be able to learn something about themselves as the go through the process and about the department. Lastly, we assume that through the group process students will realize if the position is what they are looking for and by participating in the process they will be able to attain and reproduce information in different situation as they encounter them.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! External Factors

"3!

There are few contextual events or circumstances that could affect the program in ways that may distort the evaluation. There are many factors that intersect and some that are out of the departments control. One of the largest factors reflects the preparation of students during the group process portion of interviews, which may present a challenge. Another external factor could be the amount of informational session being offered, which leads to fatigue from students and staff members. A third circumstance that could have a large effect on the evaluation process is the interviewer bias as well as questions that pertain to a specific group of students. Additionally, the lack of diversity in sessions and activities can contribute as external factors. The recruitment dramatically affects the selection process and the direction in which selection will unfold. Lastly, follow up is a very important factor. If there is no follow up it can contribute negatively to the process. These factors are able have a short term or long term effect on the process when identifying and defining what the program should and can be. Evaluation Approach (Combination) The evaluation approach that we will use for the Selection Process is formative. We chose to use this method because the area we would like to focus on is methods to improve the delivery of the selection process. An evaluation is considered to be formative if the primary focus is to provide information for program improvement (Fitzpatrick, 2004, p.16). Formative evaluation is beneficial to the selection process because of its goals to provide information for program improvement through the lens of the students at LUC. As previously stated the problems surrounding the selection process focus on unsuccessful recruitment strategies. The problem with the process is that it does not allow the department to reach the overlooked student populations. This helps the Department of Residence Life see success with recruitment but also

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"4!

the needs for improvement with the recruitment, marketing, and outreach plan. Without reaching each student population the results of the selection process lacks a diverse student staff. For the formative evaluation we are taking a combination approach of needs assessment and process evaluation. The needs assessment will determine if there is a need or problem with the delivery of the Selection Process. Fitzpatrick (2004) suggest that needs approach questions! are concerned with addressing whether a problem or need exist. After determining if a problem or needs exist the next step is making recommendations for ways to reduce the problem (Fitzpatrick, 2004, p.21). The use of the needs approach will be determined by the quantitative data collected by the Department of Residence Life. The quantitative date will reflect the applicant pool, accepted staff, and attendees of information sessions. The process approach will focus more on the delivery of the selection process. The process evaluation will allow us to determine if the outlined outcomes of the selection process are being met (Fitzpatrick, 2004, p.21). Process evaluation describes how the programs is being delivered with the following area as a focus: Nature of deliver Successes and problems encountered Quality of delivery Delivery environment Printed material

One area that will be assessed in this evaluation is the marketing, which includes informational sessions, which are mandatory for students interested in!applying for any student staff position. We will also need to determine if the timing of the informational sessions and the marketing locations create issues with the selection process. These approaches are appropriate because they will allow students to disclose areas that may need to be reevaluated and improved

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"5!

upon. We think the best way to evaluate the selection process is to attack the problem at the source, and the source in this situation is the delivery of the recruitment process and the pol of students being reached by recruitment efforts. With this approach there are strengths and weaknesses. We believe some areas of strength will be the use of information gathered from the students who attend the information sessions, participates in the interview process, and student who accept the position. This may also serve as a weakness because the information gathered may not help us in answering our evaluation question nor may it be of use. Another issue is the information gathered each year has never been analyzed for these of assessment implementation (Rutkowski, K, 2013). This may be a weakness to the approached based in the timing allotted for the selection process. If the information gathered is not consistent across all student groups this can be a weakness of the process. Some information that will help in the evaluation process that was not previously collected is demographic, gender, first generations college student. This forces us to relay on information students have decided to disclose. Some weakness of this approach focuses on the gathering of information from students participating in the Selection Process. If students decide not to provide personal contact information or if they do and the information change this presents a challenge with the evaluation of delivery. This lessens a pool of students that are represented in low numbers. If current student staffs are being questioned on topics of recruitment, the information gathered may be skewed because this student population fits the outreach process. This may present biases when evaluating this selection process. The last weakness reflects formative approach, and evaluating the process from the student perspective. If the student feels there is no an issue of diversity in their staff this could potentially change the approach from formative to summative.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Survey Population

"6!

This survey will be administered as a testing instrument following the recruitment process to students meeting the student criteria for the survey (Appendix C). The administrator of this survey will be Tyeishia Banks and Crystal Norwood, who are evaluating the Residence Life selection process in terms of marketing, interviews, and group process. The type of sampling that will be used is a criterion sampling. In criterion sampling, we will utilize a sample with criteria identified prior to a treatment or program. This approach seems to be an effective way to receive information regarding marketing, and diversity. With this approach 114 invite will be sent to current student staff members asking them to participate in the survey and focus groups. In addition this specific sampling it will allows us to sample the student population fitting the criteria to capture more data for a more effective program evaluation. Survey Design This research will utilize a cross-sectional design as we are attempting to determine the effectiveness of marketing. The foundation of the group process will give us the ability to select and hire students that are diverse but also serves as a representation of the staff population. Due to returning staff in the positions, we have the ability to assess the perceived knowledge gained from student staffs selection. This evaluation will utilize a cross sectional design as we are attempting to determine the effectiveness of the selection process. This will reflect of the marketing strategies, diversity amongst candidates and overall structure in the group process over a specific point in time; specifically directly after the staff is hired (Creswell, 2009). A cross-sectional design will assist us as we work to determine that the student staff selection process is reaching its intended outcomes, as well as creating a diverse candidate pool in relation to the student population at LUC. Cross- sectional design is used to examine trends and patterns

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"7!

across a single point of date collection (Schuh, 2009). By using this method research concludes that we will be able to pinpoint any abnormal results of the selection process and identity areas of improvement. This design will assist us in our evaluation regardless of the student staff status of returner or a new staff member. We want to avoid survey fatigue by doing one long survey verses several short surveys for the students (Schuh, 2009). There will be no comparison group because all of the students will have gone through the same student staff selection process. The survey will be structured to ask questions regarding marketing, overall satisfaction, diversity and demographics. Questions regarding marketing in relation to Resident Assistants will reflect their experience and marketing effectiveness. Is it the locations of marketing or is it the type (tabling, emails, etc.) of marketing. Specifically, the survey will contain 29 questions covering these specific target areas: marketing, diversity, overall satisfaction and demographics (Appendix C). Implementation Since we have formulated the questions, we serve as the main contacts for the distribution of the survey. In the future years, the chair or the co- chairs of the Departments selection committee, a committee that plans and implements all departments staff hiring, would oversee the survey distribution and evaluation process. Qualtrics, research software will be used to implement the survey and the committee chair would provide the company with the necessary information to effectively distribute the survey. The survey would be distributed within a week after the selection process with the expectation that the Resident Assistant population will complete the survey. To ensure that we are receiving fresh perspectives from the student populations the expectation is for the survey to be completed within two weeks of receiving the email (Appendix K). There will be an incentive

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

"8!

of $50.00 LUC bookstore gift cards for students as the survey is distributed, in efforts to gain more participants in the survey. Upon completing the survey student will be place in a raffle for the drawing of five gift cards. We (Tyeishia and Crystal) will remind and encourage students to complete the survey in regular intervals. These intervals will occur once a week for three weeks in a series of emails (Appendix K). In addition Qualtrics will notify us of who has not completed the survey after the first week to help in the reminder process for the survey. Survey Instrument The survey has twenty nine questions using radio buttons for participants to fill in Likert, interval scales, and nominal scales. This survey should take approximately 15 minutes to complete. The survey include approximately 6 student demographics questions and topic areas based on the 4 programs outcomes of the Student Staff Selection process: marketing, overall satisfaction, diversity and group process (See Appendix A). Pilot Testing A pilot test of the survey will occur before the survey is in its final form. According to Schuh (2009), pilot testing can help uncover flaws or possible biases in the instrument. In attempt to get the most useful perspectives as possible, current Loyola RAs that are returners will be asked to participate in pilot test of the survey. These students will serve as the group to recognize any significant missing components of the selection process experience, as well as the providers of feedback on area to address in the survey. These students would be the ideal candidates to test if the various questions of the survey flow naturally or need improvement (Wholey et al., 2010). Pilot testing is necessary to test for clarity and eliminate any errors in the questions.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

19!

The procedure for pilot testing the returners (RA) would go through two sessions: survey and feedback. In the first session, the board members would look over and take the survey. The second session would allow for the participants to discuss issues with the survey, allowing researchers to assess feedback for the consideration of change, clarify any miscommunications, and improve the survey. It will be beneficial because they have been through at least a year of the process. Statistical Analysis Several statistical analyses will help explore trends in data. Measures of central tendencies will be helpful as we calculate the mean, median, and mode for all continuous variables (Wholey et al., 2010). These variables are measured by interval scale on the Likerttype scale, which helps identify general patterns. The data will be looked at in relation to demographics including year in school, gender, race, and involvement within the department of residence life. As previously mentioned we will use interval scale as a means of measurement. The scale we are using is a Likert-type scales that will provide a variety of response possibilities to the questions ask in the survey. During the evaluation we will use inferential statistical analyses. In the survey we will examine gender, race, ethnicity and the effects of our marketing on the current student population at LUC. For this approach we have chosen not to compare results across a treatment and control group because we are using a criterion sampling. We chose criterion sampling because we want to review and study all cases that meet some predetermined criteria of importance as it reflects the student populations as we analyze the results. We have decided to use ANOVA as our approach to compare several different variables. With this approach we are looking to find how marketing is perceived from college students and how it

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

1"!

affects the different populations at LUC. The information we are gathering will help understand how different factors such as class standing and residential live effects our study. SPSS will serve as a tool to run analysis in descriptive and inferential ways. ANOVA test will be used to show differences in outcomes across different categorical variables. ANOVA will assist in evaluating the development of the student staff selection process in relation to three or more groups. By using independent demographic variables such as race, gender, and class standing we can assess how students perceive marketing from the department and the intended effectiveness of its use. The test will determine if differences in participants race, gender, or identities have an impact on their understanding of the departments marketing strategies. After the data is collected, Qualtrics will provide the results in bar graph form, depicting the percentage of each answer gathered from the participants. Our final report will include these graphs that contain the results of each answer presented in the survey. Utilizing this information a comparative line graph will be included depicting the levels of competence in students of color. This will also be used for the white students to determine if there is a significant difference in these levels as well. ! $:;<!<=>?@A!B;CC!D@E@>F;G@!;H!E;F@I!>@<J=>K@<I!LGD!<ELHH!L>@!M@;GN!=E;C;O@D!

@HH@KE;?@CA!HJ>!<E=D@GE!<ELHH!<@C@KE;JGP!!Q;E:!E:@<@!>@<=CE<I!B@!B;CC!M@!LMC@!EJ!;FR>J?@!H=E=>@! <@C@KE;JG<!R>JK@<<@<!EJ!@G<=>@!E:@!;GHJ>FLE;JG!;GE@GD@D!;<!D@C;?@>@D!LGD!@L<;CA!=GD@><EJJD! MA!E:@!<E=D@GE<I!LCJGN!B;E:!<E=D@GE<!LEE@GD;GN!E:@!;GHJ>FLE;JG!<@<<;JG Qualitative Approach After having the selected students complete the evaluation survey for the Department of Residence Life, on marketing regarding strategies and its importance to further engage them. Qualitative research derives its strength from gaining insight and understanding by studying a

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

11!

purposeful sample (Patton, 2002). This engagement will further discussion to ensure that we ask for and receive as much information we can about student staff selection. The qualitative approach of the evaluation will include focus groups. We are using qualitative research, the idea behind qualitative research is to purposefully select participants or sites that will best help the research to understand the problem and research question (Creswell, 2009, pg.178). In improving both the process and the outcomes of the student staff selection group process, the focus groups will ask questions to address both aspects. The answers provided from the student will be linked to the process through specific questions about satisfaction of the process, structure of the selection process, and the learning outcomes and goals set by the student staff selection committee and the Department of Residence Life. Wholey, Hatry, and Newcomer (2010) wrote that focus groups work well for evaluating programs because they give descriptive answers and explanations as to the reasons behind the results of the quantitative aspect. Focus groups allow us to collect data with questions posed in interactive group settings that will encourage participants to speak honestly and freely with other group members (Kress & Shoffner, 2007). This form of data collection will be implemented so that we can get a better understanding of the learned outcomes of the Student Staff Selection process. While the survey focuses on the marketing efforts of the Department of Residence Life and evaluates the effectiveness of selection marketing, the focus groups will be used to evaluate the outcomes achieved from the selection process. It allows for reflection of students regarding their experience of new and returning resident assistant who completed the process. The evaluations are administered to students, because of the information needed to deepen our understanding of the student staff selection process. Intended participants will be contacted via

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

12!

email. The email will serve as the invite to the focus group (Appendix K). Anticipated dates for delivery of emails will be tentative to the selection process. A series of informational emails will be sent to the focus group invitees (Appendix K-L) explaining the purpose of the focus group. This email will explain why they were selected to participate in the data selections process, and a structure for the focus group. This structure will include duration of the groups the number of expected participants, moderators, evaluators and form of transcription being used. Focus Group Participants Creswell (2009) wrote about purposefully selecting individuals who will best help the process of the evaluation. The survey will focus on all the RAs experience in the selection process. Wholey et al., (2010) wrote that focus groups participation all have something in common; Upcraft and Schuh (1996) labeled this as utilizing homogenous samples (p.56). While all the RAs been in the RA role or went through at least the process they will best able to inform us about the information throughout the selection process what needs to be improved. Using the focus groups to determine why the students were not satisfied will help improve the Student Staff Selection for the future. This would give us a pool of approximately 100 students who can be invited to be a part of the focus group but we are only selecting 30 students in total. The cap for the focus groups will be 10 participants per group, with a goal of 3 focus groups. We will split the group evenly based on availability into three groups with no more than 10 members per group with a total of 60 minutes allotted for each focus group. Our goal is to get 30 students to volunteer, giving us 100 percent volunteer rate. Participants will be contacted via email as previously mentioned with a formal focus group invitation, as noted in Appendix K. Initially, survey respondents who expressed interest in participating in focus groups will be contacted to confirm their participation, approximately 3 weeks after participating in the survey.! A second

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

13!

invitation for participates will be sent a week later. Three days following, an email will be sent to participants who responded about participating, to confirm the date, time, and location of the focus group sessions (Appendix K). Focus Group Protocol The focus group protocol instrument will be used to collect information about students experience during the selection process (Appendix D). The protocol contains the purpose of the focus group and outlines the structure of the focus group. The focus groups will be conducted on campus for participants convenience and will take place in the Regis Hall Conference room. The note- takers who will tape record and take notes during the focus groups to accurately capture everyones responses. (Appendix D). Participants will be notified that their identities will remain confidential and only the necessary Residence Life staff will use and have access to the information collected in the focus group. The structure of the focus group will include introducing of participants by stating their name, class standing, major, and which staff they are on. The questions asked during the focus group focus on the students experience during the selection process, including meaningful moments, learned outcomes, challenges, things they liked and disliked, and changes they would like to see for the process. Students are also asked to reflect on the pool of candidates and their thoughts on the pool representing the LUC student community. As previously mentioned the focus group will roughly last sixty minutes. The department will use internal departmental members to serve as moderators. This person will have experience working with the student population, but that does not serve as a direct supervisor.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Focus Group Implementation

14!

The focus groups will be implemented during the spring semester; approximately 3 weeks after participants have accepted their positions as Resident Assistants. The 3-week time frame will allow the Department of Residence Life to hire all candidates need for the positions, and to correspond with the candidates. This will give students the time to reflect on the interview process, without forgetting their experience, and emotions that came up for them during the selection group process. We will keep stakeholders well involved because, it is important to provide experiences to the administrators that will be applying the data collected to the selection process. All of the focus groups will be held on a day and time that is determined to work for all members of the focus group to ensure consistency. The focus groups will occur in the Regis Conference room, a room with a long oval table where the participants and the moderator can all be on the same level and maintain eye contact with each other. A graduate student will be the assistant moderator and two graduate students will be taking notes of the focus groups answers on a notes form designed for this focus group (Appendix E). We will use incentives listed by Wholey et al. (2010): food, in gathering participants, we will explain to the RAs that their opinions and feedback is valued greatly, hopefully making them want to provide feedback. A second incentive, lunch from Jimmy Jones will be provided for the participants. In addition to being the incentives, as the focus group environment should be comfortable (Wholey et al., 2010, p. 381), it is our hope that sitting around and eating lunch together will make the process feel like a casual conversation, leading more students to openly share their opinions with us. The session will be recorded, along with the two graduate assistant note takers in order to ensure that no information is missed, as it is important to remember

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

15!

everything that is said in a focus group (Wholey et al., 2010, p. 394). At the end of each question set, the moderator and note takers will work together to ensure that the focus groups answers were understood correctly before moving to the next section and also assist with the analysis of the data. Focus Group Analysis The focus group analysis will occur through transcriptions, and coding schemes. Creswells (2007) chapter suggests, the process of data analysis involves making sense out of text and image data (p. 183). For the focus groups we will transcribe and code for the analysis. Coding is described as organizing material such as text and making meaning of the information (as cited in Creswell, 2009). The departments moderators will need to listen to the audio recordings, and also analyze the notes taken by the note takers (Appendix E). We will utilize two writing strategies Creswall (2009) mentioned: direct quotations from participants as well as interpretations of the data (pp.193-194). In addition, a member of the department will be asked to transcribe the data collected from the recorders to ensure complete transcription of the sessions. Following this process the notes will need to be transcribed. For a more accurate coding the moderators and note takers will need to cross analysis the first focus group. For the second and third focus groups transcription will be done independently after those analyzing the data first to establish some level of inter-rater reliability to ensure coding is being done consistently. For the coding rubric, constructs will be based on prior notions of what we believe to be important area to focus on in the analysis process. The constructs are: Preparation for Position (PREP); Diversity (DER); Opportunities of Improvement (IMPV); Climate of Diversity (CLIM); Aspects of Marketing (MARK) (Appendix G). . These are constructs that have been determined before the focus group but we anticipate

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

16!

other themes and constructs will be added. Previous to the focus groups the evaluators will receive the coding rubric for their review (Appendix G). We will use the cutting and sorting technique because it allow use to develop more natural codes that emerge from the focus groups (Creswell, 2009, p. 84). Our goal is to have validity in our evaluation by using the triangulated method of a quantitative data source and a survey. This will also help in the process of justifying the themes. Limitations A few limitations exist in the qualitative evaluation design that will impact the results of the evaluation. Due to committee structure in the department and moderators it is possible that their bias or understanding or understanding of the selection process and the intended outcomes may cause a certain level of bias as they facilitate the focus groups. Since they are familiar with the intended learning outcomes for the participants, their presence during the focus group may hinder participants from being honest about their experience. The dynamic between staff members and participants may shape the types of data that is collected in the focus groups. Since there will be three different focus groups, it is likely that the direction of the conversations may alter based on the participants present in each session. It is important for each moderator to maintain a level of consistency as well as a neutral stance. To counter these limitations we will provide guidelines and tips for maintaining unbiased behavior and maintaining the conversation direction. Since the department has been in transition for the past two years with new staff members and directors, much of the rich history and knowledge of the selection process has not been archived. While we have staff members who have been in the department during its transition they may know it still lacks the data, including the perspective of the participant who did not receive a

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

17!

position. This evaluation plan begs the question; are we serving our student staff based on the needs of the residential community or what external factors and consideration are we taking into account? LOGISTICAL INFORMATION ! Timeline A timeline (Appendix H) is attached to serve as a guideline for future implication of the evaluation plan. The timing of the evaluation plan is crucial to the success of the evaluation instrument and accuracy of the data collected. Ideally with the student staff selection survey instrument and focus group all should be done within 30 days of the completion of Student Staff selection, but it is not possible due to Spring Break, preparation for Hall Closing, Room Selection, and other aspects associated with the ending of the academic year. Since time is crucial and of the essence, the most feasible timeline suggest that survey would be completed within three weeks of the end of student staff selection, and focus groups will begin within two weeks of the close of the evaluation survey. By the late May early June a rough draft of the analysis and initial report will be used to start brainstorming for the following student staff selection which occurs in February. Only a rough draft would be scheduled for this time because the department has other processes coinciding with the evaluation process that must take priority, including summer housing, capital projects, and hall opening initiatives. A final report will be completed by Mid-November when evaluators have more time to analyze the data. The final report will be presented and used to help inform the following years student staff selection for which the planning stages begin in late November. Budget The Department of Residence Life will not need to allocate a significant amount of money to the evaluation due to its access to the SPSS software, a contract with Qualtrics, and

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

18!

printing services. By using the Residence Life employees as the moderators and note takers, as well as members of the Student Staff Selection Committee as transcribers other cost are diverted as well. The majority of the $187.42 is allocated for the focus group food as displaying in the budget (Appendix I). Other cost to the department are minimal, needing only various office supplies to help conduct the focus groups. Results Presentation Qualitative data will be presented in several ways so that data can be shared with the stakeholders. This will assist in making sure the codes are outlined clearly for the stakeholders. Qualitative data will be presented in several ways so that data can be shared with stakeholders. Survey findings and data from focus groups will be consolidated and put together for the Department Selection committee. Reports will be compiled that include tables of descriptive statistics, as well as direct quotation from focus group participants. Bar graphs can also be used to show the major themes, for example the frequency that each theme emerged in the focus groups. Next Steps In a time of significant transition, an evaluation can give important insights on whether the programs goals are being met and what factors of the program may be contributing to the unmet goals. The next steps of this evaluation approach is benchmarking what similar schools are doing, and looking into other aspects that could improve group process, marketing, and overall goals. Due to the lack of learning outcomes and consistency of archive information, there is no way to compare one years process to another. An effect in implementing this plan will adjust aspects of the selection process to maximize efficiency and effectiveness as well making sure the department is hiring in relation to the student population.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! References

29!

Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed method approaches (3rd Edition). Thousand Oaks, CA: Sage Publications. Fitzpatrick, J., Sanders, J., & Worthen, B. (2003). Program evaluation: Alternative approaches and practical guidelines (3rd Ed.) New York: Longman. Fuller, W.A. (1976). Introduction to statistical inference time series .New York, John Wiley Katie Rutkowski (personal communication, September 20,2013) Kress, V. E. & Shoffner, M. F. (2007). Focus groups: A practical and applied research approach for counselors. Journal of Counseling and Development, 85, 189-195. Loyola University Chicago. (2013). Department of Residence Life. Retrieved from http://www.luc.edu/reslife/ Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage Publications. Residence Life. (n.d.) Retrieved on September 26th, 2013 from http://www.luc.edu/reslife/about/mission/ Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:Jossey-Bass Upcraft, M.L. & Schuh, J.H. (1996) Assessment in Student Affairs: A Guide for Practitioners.San Francisco: Jossey-Bass Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program evaluation (Third Edition). Jossey-Bass: San Francisco.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix A

2"!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix B

21!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

22!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

23!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

24!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

25!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix C

26!

Department of Residence Life

"#$%&'#!"#())!"&*&+#,-'!"$./&0!!
This survey will be used to improve the selection process. In years past, the Department of Residence Life has used the feedback to further enhance and improve the Student Staff Selection Process. Your feedback is important to us and will remain confidential.

Marketing:
These questions will gather information regarding your experience with the Student Selection Process with the Department of Residence Life marketing 1. Select the following position(s) that are a part of the Student Staff Selection Process? ! Resident Assistant (RA) ! Learning Community Assistant (LCA) ! Both(RA/LCA)
2. Have you ever seen marketing (tabling, flyers, emails, coffee sleeves) for the Student Staff Selection?

! Yes, I have seen marketing ! No, I have not seen marketing 3. Are you familiar with the slogan "Make Loyola Home" ! Yes ! No 4. Choose the options that best represent this logo?
! ! ! Student Staff Selection Department of Residence Life Both

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !
5. Have you seen this logo? ! Yes ! No 6. If yes, where have you seen this logo?

27!

7.

Please rate the effectiveness (successful in producing a desired result) of each form of marketing. Please rate on a scale of 1 to 4 where: 1=not effective; 2=somewhat effective;
3=moderately effective; 4=very effective

A. Flyer B. Tabling C. Informational Sessions D. Emails

1 1 1 1

2 2 2 2

3 3 3 3

4 4 4 4

8. On average how many flyer do you walk pass a day? ! 0-20 flyers a day ! 21-40 flyers a day ! 41+ flyers a day 9. Rate the likelihood of you doing the following by rearranging them in the order from 1-3. (1 being more likely 3 being less likely) ! Stopping to read a flyer ! Stopping to get a free item and a flyer ! Asking about the content on the flyer after reading 10. If the department is using flyers to market, what are the best locations for marketing? (Check all that apply) ! Residence Halls ! Dining Facilities ! Academic spaces ! Fitness areas ! Information Commons ! Student Union ! University recognized venues (Pete's, Felicias) ! Off campus venues (i.e. CVS, Five Guys) ! Other ____________________ 11. Do you believe Selection Process flyers are overlooked? ! Yes ! No

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !
12. If yes, please provide 2 reasons why flyers overlooked?

28!

13. If a department is hoping to market a position or a program what are some ways you believe is effective for the community?

Overall Satisfaction
These questions will seek to understand your overall satisfaction of the Student Staff Selection process. 14. Did you partake in the entire selection process? ! Yes ! No (If students select No, they will be redirected, and will have completed the survey) 15. Please select your overall level of satisfaction with the Student Staff Selection process " Very Satisfied "Satisfied "Neutral "Unsatisfied " Very Unsatisfied

16. Please select your overall level of satisfaction with the length of Student Staff Selection " Very Satisfied 17. Please indicate your level of agreement with the following statements about the selection process: Strongly Agree Agree Neutral Disagree Strongly Agree A. The group process was a good use of my time B. The group interview allowed me to express myself fully and the questions met my needs " " " " " "Satisfied "Neutral "Unsatisfied " Very Unsatisfied

"

"

"

"

"

Diversity
These questions will indicate your level of agreement to the following statement.
18. I felt comfortable and well informed by the senior staff member when I arrived for each interview

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

39!

" Strongly Agree

"Agree

"Neutral

"Disagree

" Strongly Disagree

19. I felt that the interview questions gave insight to how I would be supported as a staff members

" Strongly Agree

"Agree

"Neutral

"Disagree

" Strongly Disagree

20. I felt the questions asked by senior staff members in the interview and the group process allowed for a great understanding of the department value DIVERSITY " Strongly Agree "Agree "Neutral "Disagree " Strongly Disagree

21. The Student Staff Selection process allowed me to express my identities and myself " Strongly Agree "Agree "Neutral "Disagree " Strongly Disagree

22. I felt supported when I received my letter regarding the final selection decision " Strongly Agree "Agree "Neutral "Disagree " Strongly Disagree

Demographic:
The last set question that seeks to obtain the demographic of the student staff participants.
23. In which area do you currently serve as a Student Staff Member? (This information will remain anonymous) ! North-side (Bellarmine, Campion, Mertz, Fordham, Santa Clara) ! West-side (Regis, Simpson, Winthrop) ! East-side (San Francisco, de Nobili, Kenmore) ! Off-campus (commuter) 24. Select your academic standing ! Sophomore ! Junior ! Senior ! Transfer ! Graduate Student 25. Select from the following that best describes your status ! I am a first year staff member ! I am a second year staff member ! I am a third year staff member 26. What is your gender? (How do you identify) ! Female

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !
! ! Male Other

3"!

27. What is your Race/Ethnicity? ! Asian /Pacific Islander ! Black/African American ! Latino/Hispanic ! White/Caucasian ! Other 28. Based on your knowledge of student staff positions at Loyola University Chicago, which positions are available with the department of Residence Life? (Check all that Apply) ! Resident Assistants

! ! ! ! ! ! ! !

Furniture Crew Student Office Assistant Information Commons Desk worker Damen Desk Worker Learning Community Assistant Desk Receptionist/ Mangers Other (Please Specify) ____________________

29. Would you like to participate in focus group to share your ideas and thoughts regarding the Student Staff Selection? ! Yes- If so please include email: @luc.edu

! No Thank you for participating in the Student Staff Selection Process Survey! The Department of Residence Life values your feedback and will continue to improve the Student Staff Selection process for you and the department. If you have any questions, feel free to contact Crystal Norwood at cnorwood@luc.edu or Tyeishia Banks at tbanks1@luc.edu

Appendix D

Focus Group- Department of Residence Life

Focus Group Purpose: To evaluate the Student Staff selection process and the departments marketing strategies for

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

31! the process 1. To explore the satisfaction of Student Staff Selection marketing strategies 2. Explain in depth the ways in which the Selection process has prepared them for each component
a) Group process readiness b) Informational sessions c)

Focus Group Topics:

3. Identify ways in which areas of the Selection process can be improved

a) b) c) d) Group process interview Informational sessions prep Marketing Strategies Diversity in activities

Focus Group Script: Thank you all for coming today. My name is _____________and this is ________________and ______________. We are grateful for your willingness to provide feedback about the Department of Residence Life Selection process by participating in this meeting. Please feel free to grab some lunch if you havent done so already. We will be spending the next 60 minutes talking about your experience with the Student Staff Selection process. There is no right or wrong answers, and wed like to hear from everyone if possible. What is shared today will be recorded by _________ and _________and a tape recorder so we can be sure fro program evaluations to help us make improvements for future participants. We will be sharing this information with the Selection committee within the Department but not with external administrators. Please look at the consent form and sign it if you agree to the terms. Are there any questions before we begin? The evaluators have three goals 1. To explore the satisfaction of Student Staff Selection marketing strategies; 2. Explore ways in which the Selection process has prepared them for each component; 3. Identify was in which areas of the Selection process can be improved. We are going to focus on these goals in our discussion today. Lets begin by introducing ourselves. Lets go around the table- please say your name, major, and your year in school at Loyola University Chicago. Lets begin Introductory Question (5 Minutes) Lets begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on. Moderator: Thank you all very much and it is nice having you here today, so lets start with your overall impression of the marketing strategies Student Staff Selection.
10 minutes Goal 1: Marketing 1. How do you think marketing effects the process?

Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan?

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

32!

Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance) Moderator: So what I am hearing is that many of you believe marketing should look like (list them) based on what is appealing to the students. In addition, your overall satisfaction with the marketing strategies were (summarize satisfaction) and moving forward you suggest this (list them). Moving on from what went well or what did not, we now want to focus on how the application process met your needs and prepared you. 10 minutes: Goal 2: Application Process
2. What suggestions do you have for improving the application? 3. What suggestions do you have for improving the interview process?

Probe: How informed were you after attending thee informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process? Moderator: Thank you for identifying ways for improvement if any within the application process. I also heard many people say they were more comfortable with certain aspects and seen effectiveness in certain areas (list them), while others felt differently for the following areas (list them), now we will focus on aspects and ideas that will improve Application Process. 10 minutes Goal 3: Group Process
4. How supported did you feel by the graduate assistants, professionals, and returning staff members?

Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort? Moderator: In terms of improvements, you all had the following suggestions about time [list them] and ideas for how the information should be presented, such as [list]. Finally you all head helpful tips to help improve Group Process overall with [list suggestions]. Today we were able to gather your feelings on overall satisfaction of Group Process, your preparation, and ideas for improvement, we know that there may be some lingering topics you wanted to discuss, so our final questions seeks to wrap up any last statements. 10 minutes Goal 4: Diversity
5. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel?

Probe: How comfortable were you with the questions being asked by the interviewer? And why?
6. How do you see the Department of Residence life meeting the needs of these specific populations? White/ Black/Hispanic/Pacific Islander?

Probe: What do you see as challenges for staff members that identify with these identities? Moderator: Thank you for discussing areas pertaining to one of the departments values in regards to diversity. We heard many responses in regards to needs and comfort. We will now close with a final wrap and we appreciate you openly sharing how you feel in terms of diversity within our selection process.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! 5 minutes: Wrapping up Overall:

33!

7. Is there anything else related to the Student Staff Selection process that we did not cover that you would like to share?

Thank you so much for sharing your thoughts with us today, we appreciate it! We will be compiling the results of this focus group and sending it out to everyone here today to get your feedback as to whether our summary has captured your comments accurately.

Appendix E

1-#&2!3(4,'5!"6&&#7!)-.!8-+$7!9.-$:7!
1(;&<!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

34!

=(#&<! 1-#&2!3(4&.<! !"#$%&'()#*+&*,'%-.&/$%(-%&'01&',&2(/'30%&(44%2'&5#(&6%"(5#,0-&'"('&2(/'30%&'"%&(44%2'&& 4,0&%7(8/$%9&',*%.&/#'2".&4(2#($&%7/0%--#,*-&(*:&,0&+%-'30%-.&%'2;& >(.#,+,:('#7!1(;&! ?-%&7!!('%!@%&'#,)0,'5!)&(#$.&7!,)!'&&%&%! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! A$&7#,-'7B>.-;:#! 1-#&7! Intro: !

Lets begin by having you all introduce yourselves. Please state your name, class standing, major, and which staff you are on.

! ! 9-(*!C<!D(.4&#,'5!

! !

8. How do you think marketing effects the process?

Probe: What is your overall satisfaction with marketing for student staff selection? Do you have for improving the marketing strategies for the process? Probe: Who responsibility do you think it is to market these position? And who should be included in this plan? Probe: What are your thoughts regarding the marketing timeline? (Starting 8 weeks in advance) !

! !

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! 9-(*!E<!Application Process!

1. What suggestions do you have for improving the application? 2. What suggestions do you have for improving the interview process?

35! !

Probe: How informed were you after attending the informational sessions? . Probe: Do you think the informational sessions were effective in preparing you for the selection process? ! ! 9-(*!F: Group Process!
1. How supported did you feel by the graduate assistants, professionals, and returning staff members?

! !

Probe: How was your interaction in the activities in the group process? How would you describe the environment and comfort? ! ! 9-(*!G<!Diversity !
1. Do you feel there is a decrease or increase in diversity within the applicant pool? Probe: How did you feel in the interview setting? Did you feel informed by the panel?

! !

Probe: How comfortable were you with the questions being asked by the interviewer? and why?
2. How do you see the Department of Residence life meeting the needs of these specific populations? White/ Black/Hispanic/Pacific Islander?

Probe: What do you see as challenges for staff members that identify with these identities?

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! !

36!

! Wrapping up Overall:
1. Is there anything else related to the Student Staff Selection process that we did not cover that you would like to share?

! !

! !

Appendix F

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !
Consent Form Project Title: Department of Residence Life Student Staff Selection Evaluation Evaluator: (Name) Purpose:

37!

The Department of Residence Life at Loyola University Chicago is conducting a program assessment under the supervision of the Associate Director for Staff and Programs. You are invited to participate. The purpose of the study is to examine the Student Staff Selection Process. Specifically, we want to understand what learning outcome are achieved during the selection process/if the student staff reflections the overall student populations at Loyola University Chicago. We will use this information to improve the student staff selection process as we continue this process for coming years. Procedures: If you participate in this focus group, you will be in a group of approximately 8-10 students. There will be a facilitator who will ask questions and facilitate the discussion, and two note-takers to write down the ideas expressed within the group. If you volunteer to participate in this focus group, you will be asked some questions relating to your experience within your Resident Assistant or Learning Community Assistant position at Loyola University Chicago as a new or returning staff member. These questions will help us to better understand the learned outcomes of the Student Staff Selection Process. Voluntary Participation: Your participation is completely voluntary. You may withdraw from this study at any time without penalty. Benefits and Risks: Your participation may benefit you and other Loyola University Chicago students by helping to improve the selection process, experience during the process, and expected learning outcomes. No risk greater than those experienced in student staff selection process are anticipated. However, if something during the group causes discomfort, you will have received a list of campus resources where you can seek counseling. Everyone will be asked to respect the privacy of the other group members. All participants will be asked not to disclose anything said within the context of the discussion, but it is important to understand that other people in the group with you may not keep all information private and confidential. Confidentiality: Anonymous data from this focus group will be analyzed by the Department of Residence Life staff and reported to Student Affairs administrators. No individual participant will be identified or linked to the results. Study records, including this consent from signed by you, may be inspected by the administrators. The results of this focus group may be presented at departmental however; your identity will not be disclosed. All information obtained in this focus group will be kept strictly confidential. All materials

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

38!

will be stored in a secure location within the Department of Residence Life and access to files will be restricted to professional staff. Contacts and Questions: If you have any questions about the focus group you can contact (name) at (email address). Statement of Consent: By signing this consent form, you are indicating that you fully understand the above information and agree to participate in this focus group. You will be given a copy of this form to keep for our records. Participant's signature: ___________________________________________ Printed name: ________________________________________ Date: _____________________________________________

Appendix G

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

49!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix H

4"!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix I

41!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix J Email to all RAs with Qualtrics Links to Student Staff Selection Survey Dear [Insert Name],

42!

We hope that you are having a great semester and everything is going smooth. You are receiving this email because the Department of Residence Life is constantly seeking ways to develop students holistically and making every interaction and experience transformative. Since you currently represent the Department this process begins with you. One particular aspect that we are seeking to improve and get feedback is the Student Staff Selection that not too long wrapped up. In the past the Department has tried to get honest feedback. The data collected has allowed the Department to make changes in modifying length and structure of the group process day. . This year we are providing you with a new survey, which is shorter and targeted at more than just satisfaction with Student Staff Selection. We want your input Marketing Strategies and how we can increase diversity amongst the student staff. At the end of the survey, RAs will also seek the opportunity to take part in a focus group, which will help the Department of Residence Life truly understand the selection process from your perspective. The focus group will last no more than one hour and lunch from Jimmy Jones will be provided for those participating. The link to the survey is attached below. All feedback you provide will remain anonymous and will not be tied to your personal information. We ask that you be honest and constructive with your feedback. LINK Again, we truly value your input and feedback for Student Staff Selection. Should you have any questions feel to email Crystal Norwood at cnorwood@luc.edu or Tyeishia Banks at tbanks1@luc.edu Best, Crystal Norwood and Tyeishia Banks Email Reminder1: Subject: Reminder: Student Staff Selection! Dear [Insert Name], We hope that you are going well for you and your semester is going great.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

43!

Our records indicate that you have not completed the Student Staff Selection Survey. As a department, we truly value the input and feedback that you offer with your recent experience of Student Staff Selection. This is your chance to let us know what you liked and did not like about training, as well as how we can better tailor the information so it does not feel overwhelming. Please take some time to complete the survey so that we can continue to improve the experience here at Loyola University Chicago especially within the Department of Residence Life. At the end of the survey, you will be able to take part in a focus group, where you can voice your opinions more in depth. The one hour focus group will happen on a later date with food provided from Jimmy Jones. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at cnorwood@luc.edu or Tyeishia Banks at tbanks1@luc.edu Again, we truly value your input and feedback with Student Staff Selection. Best, Crystal Norwood and Tyeishia Banks Email Reminder # 2 Subject: Final Reminder: RA Fall Training Survey! Dear [Insert Name], Our record indicates that you have still not completed the Student Staff Selection survey. We would really like to stress the importance of the survey as your chance to have a large impact on future Student Staff Selection processes. This is your last chance to take part in our evaluation of Student Staff Selection. Below is the personalized link that you have received previously. Link If you feel like you received this email in error, please contact Crystal Norwood at cnorwood@luc.edu or Tyeishia Banks at Tbanks1@luc.edu Again, we truly value your input and feedback for Student Staff Selection Best, Crystal Norwood and Tyeishia Banks

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix K Focus Group Email Template 1: Email Templates for Focus Group Subject: INVITATION TO PARTICIPATE IN RESIDENCE LIFE FOCUS GROUP Hello from the Department of Residence Life

44!

The department of Residence Life invites you to participate in a lunch dialogue focus group as part of an assessment of the Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas: To explore the satisfaction of Student Staff Selection marketing strategies. Explain in depth the ways in which the Selection process has prepared them for each component. Identify was in which areas of the Selection process can be improved. The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature)

Focus Group Email Template 2: Focus Group Second Invitation Hello from the Department of Residence Life When you completed the survey for the Department of Residence Life, you told us you might be interested in doing a focus group with our office. This is our second attempt to be in touch with you about participating---we hope youre still interested in joining us! The Department of Residence Life Student Staff Selection Process. We are constantly working to improve the selection process as well as the student experience who participate in the process. The purpose of the focus group will be to address the following areas: To explore the satisfaction of Student Staff Selection marketing strategies. Explain in depth the ways in which the Selection process has prepared them for each component.

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

45!

Identify was in which areas of the Selection process can be improved. The focus group will take place in the Regis Hall Conference Room. Lunch will be provided in appreciation of your participation and the focus group will be no more than 60 minutes. Please respond to this email if you would like to participate in a focus group with your name and available dates and times by (date) from the following list: (dates) between 11:00am-12:00pm, 12:00pm-1:00pm, and 12:30pm-1:30pm. Once we receive your list of available dates and times, you will receive a final confirmation email from us with one of the dates and times you have been selected for. Please confirm your participation by (date). If you have any questions, please do not hesitate to contact (name graduate assistant) at (phone number) or (email address). We look forward to hearing your valued thoughts and opinions on your experience within the Residence Life Student Staff Selection Process. Have a great day, (Signature)

Focus Group Email Template 3: Focus Group Acceptance Email: Dear (Student Name), Thank you for agreeing to participate in this interview, this email serves as a confirmation that the focus group you will be attending will be held on (date) at (time) . The focus group will take place in the Regis Hall Conference Room. The focus group will be no longer than 60 minutes. Please let us know if there are any accommodations you may need to participate in this focus group. As a reminder you will need to complete a consent form upon participating. If you have any questions feel free to contact me at (email) or (phone number). We look forward to the focus group. Thank you, (Signature)

Focus Group Email Template 4: Focus Group Reminder for those who signed up Dear [insert name], You have signed up to participate in our focus group on [insert date] at [insert time] in [insert location]. Please arrive and we will orient you, invite you to eat, and jump in to the conversation! If your availability has changed for the focus group, please let us know by replying to this email so we can adjust our planning; we request at least 24 hours in advance if possible. Thank youwe will see you soon! Sincerely, (Signature)

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Focus Group Email Template 5: Focus group Thank you THANK YOU!

46!

Your willingness to spend the time in conversation with us is so important to being able to build on and improve the Student Staff Selection process at Loyola. Without your dedication, we would not be able to keep what you value and modify what you suggest we change. Thanks for your commitment to our residential population. Have a good rest of your semester! If you have any questions, please feel free to be in touch with (Name) at (Email)

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! ! Appendix L

47!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

48!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

59!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

5"!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

51!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

52!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

53!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

54!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

55!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

56!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

57!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

58!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

69!

#$%&'($!#$)**!#'+',$-.(!/0.,'##!! !

6"!