This month check out our new web format and read about:

Strengthening Student Success conference 2008

RP Group considering support for Small/Medium Sized Community Colleges

Cal-PASS study on CST as possible placement test for community colleges

Recent Abstracts from the CCC System Office

New CSS projects and the need for researchers

Join the RP Group Board - Nominations due by April 21

Establishing an IRB at College of the Canyons

Plus

RP 2008 Awards of Excellence

Regional Research Group Meeting in Sacramento

Drive In Workshops for SPSS and GIS

Next month, we want to end the semester with an update on all the new faces in new places within our profession. Please send me updates on newly hired research and planning professionals, including assessment coordinators (during the current year). Or let me know of advancements and retirements within the field. If you or your colleagues attend an excellent workshop or plan to present, please share this news. You can also send me open positions by the title and college. Just send a short email to mbeam@kccd.edu. Deadline is May 5 for the last issue this spring.

Planning for the annual 2008 Strengthening Student Success Conference: Beyond Basic Skills is in full swing. The RP Group partnered with the State Academic Senate this year to put on what will promise to be the most dynamic and valuable conference on student learning outcomes and assessment to date. Proposals have been submitted from around the state and the Strand Leaders reviewed the submissions to determine optimum programming for the conference. Registration opened earlier in the year and limited spaces are filling fast. If you plan to send faculty or staff to this conference we urge you to register as soon as possible. More information about the conference and registration can be found at: http://www.rpgroup.org/SuccessConf-Main.htm.

The problem of inadequate research and planning capacity has become increasingly evident at a growing number of the California community colleges, particularly small and rural institutions. The unmet need of these colleges to make proactive, data-driven decisions to improve student learning and to conduct meaningful institutional self-assessment could pose grave consequences for these colleges and for the system itself. Currently, there are about fifteen California community colleges that are on some form of warning with the Accreditation Commission (ACCJC), most of which are small to medium-sized colleges. (See, http://www.accjc.org/Actions_institutions.htm) Reasons for accreditation warnings include long-term institutional problems establishing program review systems; planning and budgeting systems; learning assessment cycles and issues related to governance.

In October 2008, the Board of the RP Group began discussion of this pressing issue, and at its winter meeting, decided to draft a letter to the community college chief executives to discuss solutions, including the creation of a ‘small college consortium’ as a way to increase research capacity and access to effective data analysis.  On April 7, the Board delivered a letter soliciting interest in this proposal to the college presidents and to the Chancellor’s Office. (See, www.rpgroup.org) To be effective, the collaborative must combine resources from the colleges in need, statewide community college organizations, and the systems office.  In particular, the RP Group letter points out the economy of scale that is possible with the creation of a small college consortium.

The underlying root of the research capacity dilemma is borne out of two increasing burdens being placed on community colleges. First, there is a greater demand on the institutions by ACCJC to provide more evidence of student learning and achievement. Second, the federal and state governments are also demanding improved data collection efforts and more thorough analysis of the indicators that demonstrate institutional effectiveness.

Larger, well-funded and favorably located community colleges have established viable and effective research and planning departments with sufficient resources and expertise to meet the increased ACCJC and governmental requirements.  However, many small and rural colleges lack the resources and personnel to meet these augmented demands for accountability.  As a result, a growing divide is being carved into of the system, potentially producing a collection of community college haves and have nots. 

The problem facing small colleges appears best addressed on two fronts. First, there must be a mechanism to provide immediate access to effective research and data collection. Second, there must be a change in culture at small and rural community colleges to recognize the critical role that researchers play and to help small institutions recruit trained research professionals.  If small colleges do not find the capacity to support all the functions related to institutional effectiveness, some type of permanent consortium of colleges will need to be established providing research, planning and program review support to the colleges.

As part of the consortium, member institutions could develop uniform and systematic methods to conduct investigations and manage data, share resources and research results, and secure professional assistance to prepare necessary accreditation and strategic planning reports. Once in place, the consortium could then establish standards and practices, arrange professional development opportunities, and create a clearinghouse for exemplary work. Additionally, the consortium could eventually help better define the role of the institutional research at small colleges.

The difficulty caused by the increased demands on small college research departments is not unique to California. In 2007, the Association of Institutional Research (AIR) is establishing two separate committees charged to investigate the status of research department effectiveness across the country and the viability of AIR to meet the institutional demands for professional services.

Research studies from across the country reveal similar concerns about the increased strain on institutional researchers, especially at colleges with limited resources.  Specifically, studies indicate that the scope and function of institutional research at small colleges vary significantly across the institutions. (Delaney, A. M. (1997) The Role of Institutional Research in Higher Education: Enabling Researchers to Meet New Challenges, Research in Higher Education, 28, 1-16.)

Moreover, institutional research job descriptions lack consistent definitions.  Further, institutional research tasks at small colleges are often relegated to mere report generation for state and accreditation agencies.  (See, Clyburn, M. (1991). An Investigation of Institutional Research in Small, Private Colleges in the Southeastern United States. Paper presented at the annual forum of the Association for Institutional Research, San Francisco, CA (ERIC Document 336 030).

Indeed, the definition of what is considered a ‘small’ college is relative and the predicament facing smaller colleges may, in fact, be a subset of the larger concern of institutional capacity at community colleges in general.   (Morest, V.S. & Jenkins, D. (2007). Institutional Research and the Culture of Evidence at Community Colleges. http://www.achievingthedream.org  Nevertheless, the tax on institutional research capacity is keenly felt at schools with limited resources and the proposal by the RP Group Board provides a viable way to tackle the problem.

The William and Flora Hewlett Foundation funded the California Partnership for Achieving Student Success (Cal-PASS) to examine whether 11th grade California Standards Test (CST) results and course grades can be predictors of the levels of a student’s first attempted community college math and English courses and the subsequent grades in those classes. If results of the CST or course grades predict college preparedness of incoming high school students, it might be possible to reduce the need for remediation at the college level by putting students on a corrective trajectory before they leave high school.

We used data from the California Partnership for Achieving Student Success (Cal-PASS) database for these analyses. Cal-PASS is a publicly funded initiative that accepts student-level course data from regionally based, intersegmental partnerships of high schools and post-secondary institutions. Data include demographics, coursework, awards, and CST scores. The CST’s were implemented beginning in 2002 and are given to students in grades 2 through 11 in various subjects. This study focused on the 11th grade English and math CST’s. For math, several different forms are available such as Algebra I (Elementary Algebra), Geometry, and Algebra II (Intermediate Algebra). The content of the CST’s are aligned with the California State Standards that guide the content of K-12 classroom instruction. The intersegmental nature of the Cal-PASS database provides an opportunity to study how CST scores and high school achievement relate to the levels of and grades in the first attempted community college math and English courses. However, the database is limited and does not have data for all California schools, and the period covered for most schools is 5 years (2001–2006).

We generated two sets of data by matching high school student records with 11th grade English or math CST scores to records with those from California community colleges that attempted English or math. For the English data set, we had 4,700 students from 24 high schools and 47 community colleges. For the math data set, we had 3,743 students from 24 high schools and 31 community colleges. Because each community college uses a separate assessment system, Cal-PASS data do not include placement test scores.

In this analysis, we examined the course level and final grade of the students’ first attempt in math or English at a community college within the Cal-PASS system. Overall, 11th grade math CST scores were better predictors of both the level and grade of the first attempted community college math course than was 11th grade math achievement. In contrast, 11th grade English CST scores were better predictors of the level of the first attempted community college English course than 11th grade English achievement, which in turn was a better predictor of grade in the community college English course than the students’ 11th grade English CST scores.

We were particularly interested in seeing if there was a specific CST score above which students were highly likely to succeed in transfer-level coursework. We created a set of ‘cut scores’ to indicate the combination of CST scores and grades that correspond to the various levels of community college English and math and their predicted success in that level. The associations between 11th grade CST scores and course grades and community college course level and success were not consistent between institutions. It is possible that differences in course content between high schools or community colleges may be a contributing factor.

Finally, a series of Classification and Regression Tree (CART) analyses were conducted to explore the influence of student and institution level factors. In most cases, the high school of origin or the community college attended were strong predictors of the course level and final grade of the students’ first community college English or math course.

We conclude that information on students’ high school grades and CST scores can be used to provide high school students with guidance on their readiness for college-level work in a fashion similar to the California State University’s (CSU) Early Assessment Program (EAP), which uses an augmented version of the CST to inform students of their readiness for CSU English and math coursework. Information on high school grades and CST scores can also assist community colleges in making placement recommendations. However, the use of these data should be strongly informed by local conditions. The value of high school grades and test scores for use in placement into community college English and math courses would likely improve greatly with increased alignment of curriculum between high schools and colleges. A draft of the report is available for download at
http://www.calpass.org/CurrentReports/LeveragingCAStandardsTest.pdf.

Although we have been heavily focused on the 2008 ARCC report for the past few months, our long-term commitment to research motivates us to share some of the material we have seen in our continuous literature review process.  If you want to scan the entire library of research abstracts that we have completed, please visit the “Biblio Trek Website” with the link provided below.

Biblio Trek Website:

http://www.cccco.edu/SystemOffice/Divisions/TechResearchInfo/ResearchandPlanning/AbstractsofResearch/tabid/298/Default.aspx

Since the February issue of this newsletter, we have added the following abstracts that you can access directly by clicking on the abstract title shown below.  These fourteen abstracts cover diverse topics that we hope interest and inform people.  Still, we invite readers’ comments and suggestions so that we can continue to assist the system’s researchers/planners.

Research Methods

• Effect Size Interpretation: Points From the MDRC
• Path of Effects: Moderators and Mediators
• Writing for a Peer-Reviewed Journal

Strategic Planning

• An Assessment of Higher Education Funding Models
• Immigrant Digital Divide
• The RAND Report on a K-20 Student Unit Record (SUR) for California
• Student Flow Analysis

Student Learning/Participation

• Factors in Completion at Four-Year Institutions: The Titus Study
• Four-Year College Student Dropout Risk and Financial Aid
• State Articulation Agreements and Probability of Transfer
• Tuition Level and College Enrollment: Shin and Milton

Workforce Development

• College Persistence by Low-Wage Workers: An MDRC Study
• Two Views of Labor Market for College-Educated
• Vocational Focus at Community Colleges  

This spring, the Center for Student Success began work on two new research projects.  The first, underwritten by the Walter S. Johnson Foundation, will investigate existing services for emancipated foster youth enrolled in California’s community college system.  CSS will seek to understand the range of programs and services available statewide, develop a report describing these programs and services, and make recommendations as to how this vulnerable population can be best served by the community college system.

A second project evaluates the Strategic Literacy Initiative, a WestEd program that helps students become more motivated and successful readers and writers by offering intensive hands-on training to instructors. Over the next two years, CSS researchers will conduct surveys and focus groups with California community college instructors who go through this program and document its impact on their teaching.

In addition to these new projects, CSS continues to work on several multi-year studies, including an analysis of transfer strategies and practices, a review of career technical education sectors and career pathways and their articulation with four-year institutions, and an assessment of a Title V Hispanic Serving Institutions grant at Mt. San Antonio College.

With the growing number of research and evaluation projects, CSS is looking for additional people to join our team. We welcome faculty, administrators, and staff with experience conducting research including designing surveys, leading focus groups, analyzing large data sets, and creating informative and actionable reports.

If you would be interested in joining our pool of researchers, please fill out the survey located at: http://www.surveymonkey.com/s.aspx?sm=Lrp1Px_2fxHUeAWpAq8Xz8Lw_3d_3d .

The RP Group is seeking nominations for five at-large officer positions - three in the North and two in the South. The RP Board offers many opportunities for professional and personal growth. You can shape professional development opportunities for your peers, become involved in statewide initiatives to further student learning, and join interesting research projects.  Furthermore, you will have the opportunity to work closely with a dynamic, funny, and creative group of professionals united by an unwavering commitment to and passion for quality research, planning, and assessment within the California community college system.

Board terms last two years, with the exception of one of the North openings, which will be for one year (this position replaces a board member who served half of a term).  Board membership entails four face-to-face meetings each year, plus involvement in organizational activities. The first meeting will be August 14 – 15 in Oakland.  The second meeting will be September 30 in Anaheim. The remaining meetings will be one day gatherings, and in each case, you will be reimbursed for your travel costs.

We are pleased to propose the following current board members for RP’s leadership positions:

President 2008-2010: Barbara McNeice-Stallard, Mt San Antonio College

Past President 2008-2009: Ken Meehan, Fullerton College

Treasurer/Secretary 2008-2010: Michelle Barton, Palomar College

Vice President – South 2008-2010: Marc Beam, Kern Community College District

Vice President – North 2008-2010: Rob Johnstone, Foothill College

Communications Officer 2008-2010: Bob Gabriner, City College of San Francisco

Conference Chair 2007-2009: Andreea Serban, South Orange County Community College District

Partner Conference Chair 2008-2010: Cathy Hasson, Skyline College

If you are interested in serving on the RP Group Board or know of colleagues with such an interest, please e-mail your nomination to Kathy Booth, RP Executive Director, at kbooth@rpgroup.org by April 21, 2008. The nomination should include your name, title, college/district, and one or two paragraphs stating your professional interests and intended contribution to the RP Group, if elected. Once received, nominations will be assembled in ballots, which will be voted on by member institutions in May 2008. The ballots will also allow members to vote on the proposed officer slate. 

As most of you know, Institutional Review Boards, affectionately called IRBs, are common in higher education and ensure that ethical standards are being met for all research involving human subjects. A common question related to IRBs is: Do community colleges need to have one? In  fact, when I came to College of the Canyons from UCLA, I asked the same question. The answer to this question, I am convinced, is “yes.”

There are several lines of reasoning for a community college establishing an IRB. First, I would argue that it is required of colleges receiving Federal money, including financial aid if the college conducts research involving people. While the absence of IRBs has gone unnoticed, I would not expect this to continue indefinitely, especially for colleges pursuing Federal grants like those through the National Science Foundation.

There is another reason to have an IRB: we need to ensure ethical protection of human subjects. Most of our research is benign and arguably could be handled with exemptions.  However, there are occasional studies that would need to be scrutinized. For example, a health survey asking students about sexual behaviors and drug use should be reviewed. If responses are identifiable and they identify criminal behavior, then these records could be subpoenaed by a court and put the subjects at risk of being prosecuted. Another example is a class project on domestic violence. In this project, the student proposes to stage an escalating domestic argument in a public place and wait to see how long it takes for someone to intervene. While the invention they are anticipating is a non-violent one, it is foreseeable that physical violence could result from the staged argument. Both of these examples are real proposals from a community college. Both require thorough review and may not be able to be approved.

In additional to Federal compliance and making the right ethical decision, IRBs can create teachable moments. Having the opportunity for faculty and students to learn about IRBs and go through the process is invaluable.

So, IRBs are a good thing. While having an IRB approved by the Feds may be a bit daunting, the process is laid out clearly on the US Department of Health and Human Services, Office for Human Research Protections, website: http://www.hhs.gov/ohrp/irb/irb_guidebook.htm. Anticipate the process taking a year or two, including training. One particularly useful online training program we use is Collaborative Institute Training Initiative (www.citiprogram.org). At about $1,250 annually, this has constituted the major cost of the effort for College of the Canyons outside of quite a bit of donated staff time. Our friends from the University of Southern California and Sinclair Community College have generously allowed us to liberally borrow their forms. In the next couple of months, we hope to have the entire process online. Of course, we would be glad to make these sites available to anyone interested in them. Hopefully, by sharing resources and talking through the process, we can ensure that we are all providing appropriate protection of human subjects, taking advantage of teachable moments, staying in compliance with Federal requirements, and even making the overall process a bit less painful.

Each year at the RP/CISOA Conference, the RP Group recognizes outstanding reports and technical applications that have been created by researchers and planners in the California Community College system.  This year, there were four projects that received an award for excellence, including the Cypress College Educational Master Plan, authored by Heather Brown and Nancy Byrnes, in the planning category and “Critical Factors Affecting the Meaningful Assessment of Student Learning Outcomes” by Jerry Somerville of Napa Valley College for the Dissertation Research Award.

Two projects were significant enough to receive awards in multiple categories.  The Accountability Reporting for Community Colleges report, which was prepared by the California Community Colleges System Office Research and Planning, received both the Award for Excellence in Research and the Award for Excellence in Technical Applications.  Members of this team included: Patrick Perry, Willard Hom, LeAnn Fong-Batkin, Catherine Liddicoat, Alice van Ommeren, Myrna Huffman, Debbie Gutierrez, Tonia Lu, Tom Nobert, and Gale Perez.

Basic Skills as a Foundation for Student Success in the California Community Colleges, a report prepared by a team from the Center for Student Success, also received two awards, the Award for Excellence in Research and the Award for Excellence in Planning.  This report, which also garnered the ACCCA Mertes Award last month, was written by Deborah Boroch, Jim Fillpot, Laura Hope, Robert Johnstone, Pamela Mery, Andreea Serban, Bruce Smith, and Robert Gabriner.

Other awards included:

Award for Achievement in Research - Using Cognitive, Motivational, and Emotional Constructs for Assessing Learning Outcomes in Student Services: An Exploratory Study.  Authored by Jerry Rudmann, Coastline Community College; Kari L. Tucker, Irvine Valley College; and Shañon Gonzalez, Coastline Community College.

Award for Achievement in Research - Math Student Success Project: Practitioner-as-Research Project.  Authored by Hannah Alford, Long Beach City College.

Award for Achievement in Planning - Guidelines for Implementing the Research Planning Agenda. Authored by the San Diego Mesa College Research Committee , San Diego Mesa College.

Award for Achievement in Planning - Enrollment Management Studies to Inform Enrollment Planning.  Authored by Jim Fillpot and Keith Wurtz, Chaffey College.

Award for Achievement in Technical Applications - Enrollment Tracker.  Created by Rick Fillman and Craig Hayward, Cabrillo College.

As part of the RP Group’s strategy to develop IR in the state, the Chancellor’s Office hosted a meeting of Sacramento area researchers on February 22.  Willard Hom, Director of Research & Planning at the Chancellor’s Office, chaired the half-day meeting.  Twelve researchers attended this inaugural meeting for the Sacramento area.  Sacramento City College, Sierra College, American River College, Yuba College, Woodland, and the Chancellor’s Office had representatives at the event.

The session began with everyone giving a self-introduction.  The next topic on the agenda was “Effective Practices for the Modern IR Office.”  The main points of that discussion dealt with (1) the administrative processes and cultures on campus that affect research and (2) some global practices. 

After lunch, compliments of the RP Board, the group discussed the issue of hiring research analysts.  The discussion generated some points to consider in a strategy to help the IR offices in their labor markets.

• The RP Group could help by creating a template of job descriptions and office structure for research offices and by marketing the value of research to college CEO’s. 
• For the long-term, colleges could begin to develop a ladder for research analysts (a “deep class”) and to promote the culture of evidence. 
• Colleges could shrink the time lag between recruitment and final selection because top candidates tend to receive multiple job offers—making it hard for community colleges to secure top candidates.
• As “baby boomers” in the research profession retire, colleges will face crises in replacing them (because pay scales may not attract the needed talent and because the hiring processes lack flexibility). 
• The labor pool does not seem to have the wealth of expertise that may have existed decades ago.  (The dilemma of multiple task areas—such as special MIS functions with special databases—compounds the perceived crisis in finding replacements for existing or new research positions.)  Some colleges may confuse the functions of MIS with those of research, possibly reducing the support for the latter in times of budget cutting.

The group then discussed basic skills research.  One researcher felt that a core issue was faculty preparation for basic skills instruction.  Another researcher felt that faculty face conflicting goals when underprepared students come to their classes (so instructors have an inherently tough task).  A third perspective in the basic skills question put the focus more upon motivation of faculty to excel at basic skills instruction rather than upon faculty training for the task.  The meeting ended with a decision that each attendee would receive the contract information of all the others to begin the networking.  A poll of the region’s researchers will help to determine the date of the next regional meeting. Members from five colleges attended. Names, left to right-clockwise in the photo above are: Catharine Liddicoat; Nelle Moffett; Jim Barr; David Waite; Kathryn Bryant; Christina Aldrich; Kevin Bray; Willard Hom; Channing Yong; and
Alice van Ommeren.  Photo by LeAnn Fong-Batkin.  Not shown: Craig Kelly and Aimee Myers.

RP is pleased to announce two workshops that address how to use SPSS and GIS for common college research tasks.  The one-day SPSS course will give you a crash course on time-saving tricks, while the week-long GIS bootcamp allows you to practice extracting information from your own college data.

SPSS 15.0 Tips, Tools, and Tricks for Beginning to Intermediate Users
May 2, 9:00 – 3:00, Fullerton College, $50 (includes lunch)
Get up to speed on the latest tips, tools, and tricks that will help you save time. Learn the basics of split files, custom tables, computing variables, automatic recode, moving information from SPSS output to Word, merging files, and helpful SPSS syntax (e.g.: SHOW N.). See how easy it is to generate large syntax files for recurring projects so that in the future your time will be cut in half. In addition to the tips that help save time, also learn how to work with the Identify Duplicate Cases command, the aggregate command for tracking purposes, and how to calculate effect size in Excel from the Independent Samples t-Test Output. Time will also be set aside to address specific topics identified by the participants.

Student & Demographic Analysis Using GIS Mapping
June 9 – 13, 8:30 – 5:00, Rio Hondo College, $100 (includes software, 4 academic credits, and a follow up online component)
This hands-on workshop provides an overview of Geographic Information Systems (GIS) fundamentals and explores various applied research methods that are commonly used by California community college researchers and planners.  Course topics include terminology, downloading and mapping census demographics, and working with Crystal Reports.  You will also learn how to obtain, spatially display, and query elements such as community demographics, political geography, student population, addresses, and college services.  Working hands-on with data from your own college, the course concludes with the opportunity to dive into an applied research project.  After the hands-on portion of the course, you will have access to an online component.  Participants will also receive a one year software license for ArcGIS.  Those who successful complete this course will receive four academic credits.

For more information and to register, visit http://www.rpgroup.org/events/workshops.html