Kettle — 安装和部署

Posted on Posted in kettle经验分享

欢迎关注天善智能微信公众号,我们是专注于商业智能BI,大数据,数据分析领域的垂直社区。 对商业智能BI、大数据分析挖掘、机器学习,python,R等数据领域感兴趣的同学加微信:tstoutiao,邀请你进入头条数据爱好者交流群,数据爱好者们都在这儿。

开头介绍

一直以来服务器是linux系统,但是感觉linux图形化不强,于是从接触kettle以来都是在windows系统操作ETL的设计和处理。现在需要在linux中查看一下kettle资源库是否连接正常,以及在linux上调度kettle的job,就需要在Linux上配置kettle环境了。

登陆-linux-(切换到kettle部署的用户下面)

第一:产看系统的java环境,cat .bash_profile 文件

从上面可以看到,在cognos用户下已经安装了java jdk,版本是1.6.0_20,我们看一下kettle的最低版本要求,编辑查看spoon.bat可以看到

java的版本需求为1.6.0_17,cognos用户下的jdk版本在此之上,说明java环境已经符合。

第二:上传文件,开始部署

1. 将data-integration.zip上传到linux服务器(利用SSH将解压过的整个文件夹上传也可以)

2. 使用unzip命令解压data-integration.zip unzip data-integration.zip -d "/mnt/kettle"(如果上传的是文件夹,忽略此步骤)

3.进入解压目录,找到./kitchen.sh

键入./kitchen.sh,看输入信息

BASH:/ kitchen.sh的:权限被拒绝

从上面可以看出kitchen.sh缺少执行权限,我们再次进入看看文件盒文件夹的权限

从上面可以看出,此目录下面的.sh文件都没有执行权限(X),接下来我们给这个文件夹下面的所有.sh文件赋执行的权限

由上面的我们可以看出,赋权成功!

再次./kitchen.sh,回车 看输出信息–

可以看到上面出现 -rep,-user,-pass 这些kettle中的帮助信息,说明kettle部署成功!

第三:调出spoon界面

在windows系统上面是执行spoon.bat 同理在linux系统上面执行spoon.sh调出spoon的界面进行开发。

执行./spoon.sh,查看信息

这里说一下,调出spoon界面需要linux客户端支持图形化服务。我们这里采用的是SecureCRT_CN,只支持命令后台。这个时候就需要用tvnviewer,需要在linux服务端装tvnviewer服务端。

在命令客户端执行:vncserver 打开vnc服务

从上面的信息可以看出,服务已经启动,我们可以登陆tvnviewer

登陆成功 —

在终端执行调出spoon界面的命令!执行 xhost +(xhost + 是使所有用户都能访问Xserver.)

OK ,所有host用户都可以访问图形服务了。

进入目录,执行spoon.sh

弹出下面的信息

解决上面的问题,jdk已经没问题了到网上搜了一下

thanks 这位兄弟给出此方法

从英文的大概可以看出来是需要编辑.kettle目录下的一个文件,于是发现linux上面cognos根目录下没有.kettle文件

于是从window上面把C:\Users\Administrator\.kettle中的文件.spoonrc.kettle编辑修改

保存,然后把.kettle整个文件夹赋值到cognos根目录下,查看是否存在(使用 ls -a显示所有对象,因为.XX的对象默认是隐藏的)

再次执行./spoon.sh

[cognos@bitic data-integration]$ ./spoon.sh

/home/cognos/pdi-ce- 4.20-stable/data-integration

INFO 11– 11145634164– Spoon – Logging goes to file: ///tmp/spoon_66d28e63- 4a9e- 11e3-a301- 7b09d1d32e5b. log

INFO 11– 11145635641– Logging to org.slf4j.impl.JCLLoggerAdapter(org.mortbay. log) via org.mortbay. log.Slf4jLog

INFO 11– 11145635646– class org.pentaho.agilebi.platform.JettyServer – WebServer.Log.CreateListener localhost: 10000

INFO 11– 11145635647– jetty- 6.121

INFO 11– 11145635703– NO JSP Support for/pentaho, did notfind org.apache.jasper.servlet.JspServlet

INFO 11– 11145639221– Initializing Spring root WebApplicationContext

INFO 11– 11145639221– Root WebApplicationContext: initialization started

INFO 11– 11145639242– Refreshing org.pentaho.platform.web.http.context.PentahoSolutionSpringApplicationContext @e73783: display name [Root WebApplicationContext]; startup date [Mon Nov 11145639CST 2013]; root of context hierarchy

INFO 11– 11145639271– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/pentaho-spring-beans.xml]

INFO 11– 11145639285– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/pentahoSystemConfig.xml]

INFO 11– 11145639298– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/adminPlugins.xml]

INFO 11– 11145639305– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/systemListeners.xml]

INFO 11– 11145639310– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/sessionStartupActions.xml]

INFO 11– 11145639316– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/applicationContext-spring-security.xml]

INFO 11– 11145639331– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/applicationContext-common-authorization.xml]

INFO 11– 11145639336– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/applicationContext-spring-security-memory.xml]

INFO 11– 11145639343– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/applicationContext-pentaho-security-memory.xml]

INFO 11– 11145639351– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/pentahoObjects.spring.xml]

INFO 11– 11145639527– Bean factory forapplication context [org.pentaho.platform.web.http.context.PentahoSolutionSpringApplicationContext @e73783]:org.springframework.beans.factory.support.DefaultListableBeanFactory @da52a1

INFO 11– 11145639607– Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory @da52a1: defining beans [systemSettingsService,org.pentaho.platform.engine.core. system.SpringPentahoSystemProxy #0,administrationPluginsList,systemListenersList,sessionStartupActionsList,filterChainProxy,authenticationManager,loggerListener,basicProcessingFilter,basicProcessingFilterEntryPoint,requestParameterProcessingFilter,requestParameterProcessingFilterEntryPoint,pentahoSecurityStartupFilter,anonymousProcessingFilter,anonymousAuthenticationProvider,httpSessionContextIntegrationFilter,logoutFilter,securityContextHolderAwareRequestFilter,httpSessionReuseDetectionFilter,exceptionTranslationFilter,authenticationProcessingFilter,authenticationProcessingFilterEntryPoint,httpRequestAccessDecisionManager,filterInvocationInterceptor,roleVoter,daoAuthenticationProvider,userDetailsService,userMap,userMapFactoryBean,passwordEncoder,userRoleListEnhancedUserMapFactoryBean,inMemoryUserRoleListService,pentahoUserRoleListService,ISolutionEngine,ISolutionRepository,ISolutionRepositoryService,IContentRepository,IAuditEntry,IUITemplater,file,vfs-ftp,contentrepo,IAclPublisher,IAclVoter,IVersionHelper,ICacheManager,IConditionalExecution,IMessageFormatter,IDatasourceService,IPasswordService,IPluginProvider,IPluginManager,IPluginResourceLoader,IMenuProvider,IServiceManager,ITempFileDeleter,gwtServiceManager,IChartBeansGenerator,systemStartupSession,connection-SQL,connection-MDX,connection-XML,connection-HQL,IThemeManager]; root of factory hierarchy

INFO 11– 11145639687– Adding user [org.springframework.security.userdetails.User @c24ef000: Username: suzy; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, cto, is]

INFO 11– 11145639688– Adding user [org.springframework.security.userdetails.User @e9f02600: Username: pat; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, dev]

INFO 11– 11145639688– Adding user [org.springframework.security.userdetails.User @7ba0e400: Username: joe; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Admin, Authenticated, User, ceo]

INFO 11– 11145639688– Adding user [org.springframework.security.userdetails.User @6a247400: Username: tiffany; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, dev, devmgr]

INFO 11– 11145639745– Validated configuration attributes

INFO 11– 11145639749– Adding user [org.springframework.security.userdetails.User @c24ef000: Username: suzy; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, cto, is]

INFO 11– 11145639749– Adding user [org.springframework.security.userdetails.User @e9f02600: Username: pat; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, dev]

INFO 11– 11145639749– Adding user [org.springframework.security.userdetails.User @7ba0e400: Username: joe; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Admin, Authenticated, User, ceo]

INFO 11– 11145639749– Adding user [org.springframework.security.userdetails.User @6a247400: Username: tiffany; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: Authenticated, User, dev, devmgr]

INFO 11– 11145639761– Root WebApplicationContext: initialization completed in 539ms

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forIRuntimeRepository

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forISubionScheduler

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forISubionRepository

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forIUserFilesComponent

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forIBackgroundExecution

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forIScheduler

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forINavigationComponent

INFO 11– 11145639778– misc-org.pentaho.platform.engine.core. system.PentahoSystem: Could notdetemine implementor forIUserSettingService

INFO 11– 11145639810– Mondrian: properties loaded from 'file:/home/cognos/pdi-ce-4.2.0-stable/data-integration/libext/mondrian/config/mondrian.properties'

INFO 11– 11145639810– Mondrian: loaded 0systemproperties

INFO 11– 11145639884– Refreshing com.pentaho.analyzer.content.px @106e02e: display name [com.pentaho.analyzer.content.px @106e02e]; startup date [Mon Nov 11145639CST 2013]; root of context hierarchy

INFO 11– 11145639885– Loading XML bean definitions from class path resource [beans.xml]

INFO 11– 11145639921– Bean factory forapplication context [com.pentaho.analyzer.content.px @106e02e]:org.springframework.beans.factory.support.DefaultListableBeanFactory @f67d81

INFO 11– 11145639940– Loading properties file from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/analyzer/analyzer.properties]

INFO 11– 11145639944– Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory @f67d81: defining beans [propertyConfigurer,properties,analysisAreaManager,olapMetaDataManager,olapConnectionManager,reportManager,reportRequestService,usageTracker,localizationService,CVAppContext]; root of factory hierarchy

INFO 11– 11145639948– Loading properties file from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions/ system/analyzer/analyzer.properties]

Pentaho BI Platform server is ready. (Pentaho Platform Engine Core 3.90-stable. 47240) Fully Qualified Server Url = http: //localhost8080/pentaho/, Solution Path = /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/platform/pentaho-solutions

INFO 11– 11145640048 – Started SocketConnector @localhost10000

INFO 11– 11145640054– Refreshing org.springframework.context.support.FileSystemXmlApplicationContext @bf1a4a: display name [org.springframework.context.support.FileSystemXmlApplicationContext @bf1a4a]; startup date [Mon Nov 11145640CST 2013]; root of context hierarchy

INFO 11– 11145640054– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/visualizations/prpt/plugin.xml]

INFO 11– 11145640070– Bean factory forapplication context [org.springframework.context.support.FileSystemXmlApplicationContext @bf1a4a]:org.springframework.beans.factory.support.DefaultListableBeanFactory @9e4dc0

INFO 11– 11145640071– Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory @9e4dc0: defining beans [visualizationBean]; root of factory hierarchy

INFO 11– 11145640079 – Refreshing org.springframework.context.support.FileSystemXmlApplicationContext @7348e: display name [org.springframework.context.support.FileSystemXmlApplicationContext @7348e]; startup date [Mon Nov 11145640CST 2013]; root of context hierarchy

INFO 11– 11145640,08 0– Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/visualizations/report-wizard/plugin.xml]

INFO 11– 11145640,094 – Bean factory forapplication context [org.springframework.context.support.FileSystemXmlApplicationContext @7348e]:org.springframework.beans.factory.support.DefaultListableBeanFactory @ffb35e

INFO 11– 11145640,095 – Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory @ffb35e: defining beans [visualizationBean]; root of factory hierarchy

INFO 11– 11145640,097 – Refreshing org.springframework.context.support.FileSystemXmlApplicationContext @153c9c7: display name [org.springframework.context.support.FileSystemXmlApplicationContext @153c9c7]; startup date [Mon Nov 11145640CST 2013]; root of context hierarchy

INFO 11– 11145640,098 – Loading XML bean definitions from file [ /home/cognos/pdi-ce-4.2.0-stable/data-integration/plugins/spoon/agile-bi/visualizations/analyzer/plugin.xml]

INFO 11– 11145640123– Bean factory forapplication context [org.springframework.context.support.FileSystemXmlApplicationContext @153c9c7]:org.springframework.beans.factory.support.DefaultListableBeanFactory @3eeab3

INFO 11– 11145640125– Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory @3eeab3: defining beans [visualizationBean]; root of factory hierarchy

INFO 11– 11145640224– Spoon – Asking forrepository

INFO 11– 11145640347– RepositoriesMeta – Reading repositories XML file: /home/cognos/.kettle/repositories.xml

INFO 11– 11145641118– Version checker – OK

——————————————–出现上面的加载信息,图形界面成功调出—————————-

小结:至于解决方法ShowWelcomePageOnStartup=N 欢迎页面不显示就可以加载处设计页面,就不知道是上面原因了,此时我们在spoon界面上点击显示欢迎界面的话又会出现同样的错误,哪位大神看到的话,欢迎留言科普!